Network News

X My Profile
View More Activity


Posted at 11:00 AM ET, 08/23/2010

Willingham: 3 key factors in teacher evaluation (beyond the hype of value-added)

By Valerie Strauss

Cognitive scientist Daniel Willingham, a psychology professor at the University of Virginia, follow up on his look last week at the Los Angeles Times project that evaluated teachers by using test score data. Willingham is the author of “Why Don’t Students Like School?”

By Daniel Willingham
Last week I criticized the Los Angeles Times for publishing a story that labeled teachers as “highly effective”or “poorest-performing” solely on the basis of a value-added model of student test scores.

I mentioned that the reliability and validity of value-added models is controversial.

Setting aside for a moment the need for a reliable and valid measure, what other considerations ought to come into play when thinking about teacher evaluation?

First, an evaluation system ought to be motivated by one’s goals for schooling. What are we hoping students will get out of school? If teachers are to be held responsible for student outcomes, the sensible thing to do is to begin with the target outcomes, and then use assessments appropriate to those outcomes.

Some target outcomes will likely be hard to measure: that students will have a positive attitude towards learning, for example. But if we begin by listing everything we would ideally want to measure, we will at least have a clear idea of what we can put in an evaluation measure and what we cannot. And we may find ways of satisfactorily measuring constructs we had not considered measurable.

If we start with a test because we have it around for another purpose, we’re doing things backwards. We’re starting with the measure, and then saying “this is important to schooling.”

Second, I think that evaluations should vary for teachers of children at different ages. To my mind, it’s unreasonable to point one’s finger at the teacher of a high school senior and say, “Dan didn’t learn much math this year. That is inevitably your fault.” Where is Dan’s responsibility in this scenario?

I think it is much more reasonable to suppose that a kindergarten student is not responsible for his or her learning. Whether the parents bear some responsibility might be open for discussion.

Third, an evaluation system must confront the issue of the imperfection of any evaluation system.

To take the simplest example, suppose your evaluation amounts to a categorization: “teachers good enough to stay” versus “teachers bad enough to fire.” Two types of errors are possible: retaining bad teachers and firing good teachers.

Either mistake is costly, but you can’t withdraw from evaluation for fear of making a mistake. Even if you do nothing, you’ll still be making mistakes. If you fail to evaluate teachers at all, you’ll never fire a good teacher, but you’ll retain bad teachers.

That’s close to where we are now.

You can’t eliminate errors but you can choose the proportion of each type of error committed. You do that by varying the threshold for firing.

The proportion of each type of mistake that you elect to make would be based on your evaluation of how costly each type of mistake is.

Even with a reliable and valid measure, I’d be leery of labeling teachers “highly effective” or “poorest performing” with a system that did not account for these other three factors.

Randi Weingarten, president of the American Federation of Teachers said that teacher evaluation “has been broken for years” and I don’ t know anyone who would disagree.

But history ought to tell us that rushing to a new system would be a mistake. There is not a lot of patience in education policy for the continual refinement of policies. They are put in place and are deemed a success and retained, or deemed a failure and jettisoned.

If a new teacher evaluation system is above some threshold of perceived effectiveness, the basic structures will likely be calcified into practice and unchanged for years, even if it could be improved. We’ll rush on to tweaking some other part of the system.

If the new system is below that threshold, people will recoil from the idea of evaluating teachers (it will be the No Child Left Behind of its day) and it will be that much harder to get any effort off the ground for at least a decade.

There is unprecedented momentum for teacher evaluation. It would be a pity to waste it.

-0-

Follow my blog all day, every day by bookmarking washingtonpost.com/answersheet. And for admissions advice, college news and links to campus papers, please check out our Higher Education page at washingtonpost.com/higher-ed Bookmark it!

By Valerie Strauss  | August 23, 2010; 11:00 AM ET
Categories:  Daniel Willingham, Guest Bloggers, Teacher assessment, Teachers  | Tags:  daniel willingham, factors in teacher evaluation, how to evaluate teachers, no child left behind, teacher evaluation, teacher evaluation systems, teachers and value added, value added systems  
Save & Share:  Send E-mail   Facebook   Twitter   Digg   Yahoo Buzz   Del.icio.us   StumbleUpon   Technorati   Google Buzz   Previous: How to give classrooms a mission
Next: The most expensive public school in America

Comments

All your points were great but two in particular have been overlookd.

"If we start with a test because we have it around for another purpose, we’re doing things backwards. We’re starting with the measure, and then saying 'this is important to schooling.'”

I'd add onto your second great point that "that evaluations should vary for teachers of children at different ages."

Tests are different for different age students Why? Tests and their measurements are political compromises. We have many different types of tests for many different ages of kids because they were devised at different times during different types of political climates.

Also, policies like curriculum alignment are more or less appropriate for students of differents ages. By high school horizontal curriculum alignment is much more appropriate and it does not lend itself to the same evaluation instruments.

Also,different evaluations for different ages applies to subjective evaluations. People with different personalities choose to go to different types of schools with different cultures. Not only are primary school teachers and administrators different than educators who have seniors, but AP and IB teachers in elite schools are different than inner city teachers.

And people who teach different subjects have different personalities and create different cultures. Professor Emeritus Canady says it best because he uses humor. Math teachers often will go on and on about how they calculate grades. Social studies teachers, like me, will say "that grade looks good enough."

Reformers can go on a tirade against that reality, but its not going to change. Nor should it. I'm always going to be at my best when I'm allowed to build on my strenghts, and if I'd wanted to be a 1st grade math teacher, I'd gone into that field. I'm a former academic, and I could be a good IB teacher in an elite school, but I really love the adrenalin rush of the toughest schools.

I'm reminded of our grad school intermural b-ball games. At Rutgers at least, the English and the Comparative Literature departments could not have had teams that were more different. The Law School teams fouled at lot more than the Sociology teams. The theorectical Math teams were much more competitive than the History teams. Why would someone want us all to become the same square pegs?

Posted by: johnt4853 | August 23, 2010 11:56 AM | Report abuse

Well written and interesting comment john4853, But I promise you, while it's true my sons ages 6 and 10 do indeed have different personalities and different specific educational needs and interests; they do not have the ability to choose different types of schools, nor do I on their behalf. Willingham had a article "
What Happens to School Choice if People Aren’t Rational and Choose Bad Schools?"
People do act irrationally, do choose bad schools. I do know a couple of outcomes, bad teachers profit, bad method hucksters profit, and children's time and efforts are wasted- as if they are a by-product instead of the reason for schools.
Please let's all agree to push for great teacher evaluations in public education!

Posted by: WorriedParent | August 23, 2010 12:56 PM | Report abuse

The New York Times recently had Op-Ed Columnist from a black columnist on the problems of blacks in areas such as the Title 1 poverty public schools such as D.C.

Too Long Ignored
By BOB HERBERT

"More than 70 percent of black children are born to unwed mothers."

Time to fully understand that the problem is large numbers of children that have great difficulty in learning and may never be able to learn.

Multiple generations of poverty where the norm is almost six years of neglect before a child is placed in the public schools.

Overwhelming evidence of the tremendous importance of the first five years in the development of children.

National tests that for years indicate large majorities of the Title 1 poverty public school children with great difficulty in learning.

And the answer to the problem is supposedly evaluating teachers that are supposedly responsible for simply overcoming the years of neglect of these children and multiple generations of poverty.

There is no problem in the evaluation of teachers. The problem in public education is the refusal to recognize the epidemic in public school education in the Title 1 poverty public schools with large number of children that have great difficulty in learning and may never be able to learn.

The need to evaluate teaches in regard to this epidemic makes as much sense as telling parents to intentionally neglect their children as the teachers are responsible for overcoming this neglect.

Posted by: bsallamack | August 23, 2010 2:10 PM | Report abuse

People do act irrationally, do choose bad schools. I do know a couple of outcomes, bad teachers profit, bad method hucksters profit, and children's time and efforts are wasted- as if they are a by-product instead of the reason for schools.
Please let's all agree to push for great teacher evaluations in public education!

Posted by: WorriedParent
..................................
You have bought into the kool aid. If your public school is other than the Title 1 poverty public schools, there is no problem.

The Title 1 poverty public schools are the problem schools and not the majority of public schools in this nation that are not poverty schools.

These poverty public schools are pretending the problem is teachers while these schools hire individuals to teach with a B.A. and a 5 week training camp in teaching.

With limited training, Teach for America recruits play expanding role in schools

http://www.washingtonpost.com/wp-dyn/content/article/2010/08/22/AR2010082202893.html?sid=ST2010082301888

Posted by: bsallamack | August 23, 2010 2:22 PM | Report abuse

"People do act irrationally, do choose bad schools. I do know a couple of outcomes, bad teachers profit, bad method hucksters profit, and children's time and efforts are wasted- as if they are a by-product instead of the reason for schools."

Actually, that describes the status quo at the non-title-1 public school in my community. I don't know that choice would make the situation worse, even if it doesn't make it better.

Posted by: hainish | August 23, 2010 4:21 PM | Report abuse

Thanks again Valerie and Dan for making sense out of that which does not make sense (rating teachers by test scores).

Posted by: educationlover54 | August 23, 2010 4:40 PM | Report abuse

Teachers need to stop drinking the kool aid.

D.C. national test 2009
56 percent failure reading 4th grade.
27 percent basic reading 4th grade.
12 percent proficient reading 4th grade.
5 percent proficient reading 4th grade.

White children in the public school system for years have had the HIGHEST scores in national tests in the nation while black students in the same public school system have had the LOWEST scores in national tests in the nation.

This is the same public school system and only those who are drinking Kool aid would believe that it is a problem caused by teachers and not that the black children in D.C. have great difficulty in learning because of multiple generations of poverty.

Posted by: bsallamack | August 23, 2010 6:21 PM | Report abuse

What I was getting at in my previous comment was I understand developing great teacher evaluations may be tricky and complex, but it can be done and should be done. I read in another article how everyone, who cares to know, knows who the great teachers are at a school and maneuver their kids to the great teacher. Eh, I doubt it.

Take this example- Small charter school my kids attended for two years. I enrolled them based on the promises of the charter- Twenty students per teacher and laptop computer lab for all students.
Okay, school didn't have the laptops until the third trimester of the first year. Then the school started using "Whole Brain Teaching/Power Teaching" school-wide. This teaching "method" is so stupid, so indifferent so bad.. well, here's just one example of their writing/explanation on learning-

"Some students talk easily, often too easily! Other students fall into the role of passive listeners. In terms of brain structure, classes are often divided between those who are Brocaians (speakers) and Wernikites (listeners). By using Switch, an instructor can easily teach listening skills to the speakers and speaking skills to the listeners."

Most parents are unaware of what junk it is. The teachers just say "It's research based, it was developed for college students, it's fun and engaging.
Then- the next year, gone was the twenty to one teacher promise. Then I started counting how many children from my younger son's classes were retained- of twenty in two years- seven retained and one they wanted to but couldn't.
Finally I pulled my kids out. But not before the school got tens of thousands of dollars from state and federal funds to teach just my two kids. (Changing schools has been very stressful for my kids)

If you look on the Great Schools website for this school (http://www.greatschools.org/california/san-bernardino/24716-Soar-Charter-Academy/#from..HeaderLink) there are more glowing parent reviews than there are students. There are also last years test scores, showing all third graders did significantly worse on 2010 test than they did as second graders in 2009. I do believe those test scores reflect the poor teacher quality and idiotic methods used to teach. But I have a vault of background knowledge too. And, importantly- the test scores are at direct odds of what the claims are of the charter, the teachers, the vocal parents. And well, the numbers don't lie.


Posted by: WorriedParent | August 24, 2010 1:04 PM | Report abuse

There are also last years test scores, showing all third graders did significantly worse on 2010 test than they did as second graders in 2009.
Posted by: WorriedParent
............................
This is the problems with these tests and the focus on testing.

The real measure of a primary school is whether children enjoy reading or not.

Public schools used to be a slow process. Now it is a factory.

Children get meaningless homework since this is the current fetish.

If you want to blame anyone blame people like Jay Mathews who proclaims the answer is standardized testing in every grade and subject when we know already there are large numbers of students that can not read.

Posted by: bsallamack | August 24, 2010 3:27 PM | Report abuse

I'm not crazy about standardized testing either but I think we are all missing a key element here. Which teachers in your school systems care for the students they teach? Which ones take extra time to wipe away tears, hug a child, offer them their lunch when they don't have any, make teaching a positive, fun, experience and engender a longing to read and learn. That's who I want teaching my kids. My son had a teacher in 3rd grade who threw shoes, screamed and blamed the kids for her failures, she claimed my son who is extremely shy (even now as a teenager) was a "bad" kid and then offhand made the comment that oh by the way he might need glasses in the 3rd quarter of the school year! She didn't think it was important to mention before the year was almost over?!! When I went to the principle to complain she continued to try to justify her existence with blame on her teaching assistant who was the only teacher my son liked! The Assistant wiped tears, gave food, mentored kids and showed them faith and love and she's the one the school lost! I yanked my son out of this bad teacher's class at the end of the school year with 4 weeks to go because I couldn't believe the school would endorse her bad behavior and my son loved the new teacher because while strict she was fair with set rules and standards of behavior. We have to listen to our kids, to really hear what they say when kids like or dislike a teacher especially at a young age, they generally like or dislike teachers for a reason because kids of that age don't understand hate or chaotic teaching methods. Perhaps we need cameras in the classroom to study how a teacher interacts with students, are they abusive, ignoring the needs of the least of their students or are they patient, caring and focused on the needs of the children they care for?

Posted by: TGriggs1 | August 25, 2010 9:43 AM | Report abuse

The comments to this entry are closed.

 
 
RSS Feed
Subscribe to The Post

© 2010 The Washington Post Company