Network News

X My Profile
View More Activity


Posted at 6:30 AM ET, 06/ 8/2010

Study: N.Y. teacher performance pay program flops

By Valerie Strauss

Just the other day we heard that a program in Chicago that attempted to link teacher pay with student standardized test scores wasn’t working, at least not in the first two years.

A 2009 analysis of a major program in Texas that also linked teacher pay to student achievement gains on tests showed no evidence of success. The multi-year Texas Educator Excellence Grant involved teachers at about 1,000 campuses, with a total of more than 140,000 students in lower-income neighborhoods. It was discontinued because of “design problems.”

Now, a paper prepared by two Columbia University researchers for a recent education conference at Harvard University said that the New York City Bonus Program, which attempts to raise student achievement by paying teachers for it, was -- you guessed -- also unsuccessful.

The researchers, Sarena Goodman and Lesley Turner, investigated the impacts of group-based incentive pay over two academic years (2007-2008 and 2008-2009) on a range of outcomes that included teacher effort, student performance in math and reading, and classroom activities. Also evaluated were impacts on the market for teachers by examining teacher turnover and the qualifications of newly hired teachers.

“Overall, we find the bonus program had little impact on any of these outcomes,” the researchers concluded.

In each of these reports, the authors noted that the design of the program was flawed in some way. Teachers weren’t paid enough or they were not paid individually or some other part was not well conceived.

The Goodman/Turner study says, for example, “We argue that the lack of bonus program impacts can be explained by the structure of the bonus program. Group bonuses led to free-riding, which significantly reduced the program’s incentives.”

Maybe. But maybe not. And this is our problem: Nobody yet knows.

That hasn’t stopped performance-based pay from becoming the new mantras in school reform. It was, in fact, a key part of the new teachers' contract negotiated in Washington, D.C., by Schools Chancellor Michelle Rhee and just ratified by members of the Washington Teachers' Union. It is also part of a number of school reform laws recently passed in various states to win favor -- and federal Race to the Top dollars -- from Education Secretary Arne Duncan.

The thinking goes something like this: Why shouldn’t student achievement be included in the evaluation and compensation of teachers? For one thing, teachers aren’t the only players in the complex process in which kids learn. Home environments and biology play a role, too.

If, however, teachers are to be evaluated by how well their students do, the worst possible way to do that would be using student scores on standardized tests. There are a lot of reasons a student might do poorly on one of these tests, and how effective there teacher is or isn’t may well not be one of them. Besides, these tests aren’t designed to evaluate how well teachers do their jobs, and text experts will tell you that an assessment is only valid for the purpose for which it was designed.

Performance pay linked to test scores creates incentives for teachers to essentially do the wrong thing: Obsess on teaching kids how to do well on the tests -- in math and reading -- while giving short shrift to other vital subjects. So even if this scheme were to "work," it wouldn't really be working.

I do not suggest that these three reports are at all definitive. But they certainly show that education reformers are pushing school systems -- yet again -- into another costly experiment that may be doomed to fail.

You would think that after the disappointing results of No Child Left Behind, the folks behind school reform would think twice about jumping onto another racing train without knowing where it is going.

Unfortunately, you’d be wrong.

Follow my blog all day, every day by bookmarking washingtonpost.com/answersheet. And for admissions advice, college news and links to campus papers, please check out our new Higher Education page at washingtonpost.com/higher-ed Bookmark it!

By Valerie Strauss  | June 8, 2010; 6:30 AM ET
Categories:  No Child Left Behind, Performance pay, Teachers  | Tags:  d.c. teachers contract, merit pay, michelle rhee, nclb, performance pay, teachers, teachers performance pay, test scores and pay  
Save & Share:  Send E-mail   Facebook   Twitter   Digg   Yahoo Buzz   Del.icio.us   StumbleUpon   Technorati   Google Buzz   Previous: Standardized testing follies in Florida
Next: Your kid’s friends matter more than you think, study says

Comments

Thanks for keeping us informed, Valerie. I always appreciate the information you give us.

Posted by: jlp19 | June 8, 2010 8:32 AM | Report abuse

Valerie, how about giving equal play to information that doesn't support your personal opinion: a recent evaluation of Denver's ProComp system "suggests that ProComp has helped to contribute to an increase in student achievement over the last four years, that teachers and principals alike feel positively about ProComp, and that ProComp has helped to attract and retain high-performing teachers, particularly at our highest-needs schools." The whole report is here:

http://static.dpsk12.org/gems/newprocomp/ProCompOutcomesEvaluationApril2010final.pdf

Posted by: gideon4ed | June 8, 2010 9:22 AM | Report abuse

Gideon4ed,

The fact that you are only able to cite one example of a school district that has been successful with performance pay when there are dozens who have tried it is very telling. The bottom line is that there are so many other factors to teaching than test scores. For the past two years, test scores have supposedly improved in DC. Yet are the schools any better? Why are there still so many discipline problems? Why aren't more high school graduates from DC attending college? What about the DC schools who were suspected of cheating in 2007? Which schools are cheating and which ones are making legitimate gains? So far, increases in test scores have not provided any proof that the schools and it's tudents are actually improving, so why are we linking them to teacher pay?

Posted by: thebandit | June 8, 2010 9:48 AM | Report abuse

There is no mystery here. This doesn't work because the best teachers are motivated by student success and good working conditions (including lack of interruptions, back up on discipline issues, well run school, collegiality of coworkers etc.)
Some teachers are probably motivated by money. some are not. Then, you have to figure out how to motivate the students to do well so that you get more money. Some kids won't buy it.

Posted by: celestun100 | June 8, 2010 10:20 AM | Report abuse

I see a clear pattern of rushing to judgement, seemingly to further a personal agenda. I also see a pateern of under-reporting on data that conflict with said opinion.

Posted by: HappyTeacher | June 8, 2010 10:29 AM | Report abuse

I see a clear pattern of rushing to judgement, seemingly to further a personal agenda. I also see a pateern of under-reporting on data that conflicts with said opinion.

Posted by: HappyTeacher | June 8, 2010 10:30 AM | Report abuse

The biggest flaw with all of these programs is they way they pit teachers against one another. Most teachers are, by nature, collaborative and will work together to improve schools and curriculum if given the chance. This includes raising test scores. For example, in many schools, teachers combine small groups of students with similar learning problems for tutoring sessions for the purpose of improving performance.

When financial rewards for test scores are doled out in such a way as to give a reward only to the teacher of record, it discourages working together. It isn't fair to ask a teacher to tutor a group of low performers from other classes when someone else gets the money for the improved scores.

Posted by: aed3 | June 8, 2010 12:33 PM | Report abuse

Even the Bush administration's strongest proponents of NCLB and testing, Diane Ravitch and Russ Whitehurst, have now turned against testing children as a means of evaluating teachers, yet this insane concept continues, thanks to well-funded proponents of teach-to-the-test, scripted instruction, delivered by short-term interns with minimal training (the KIPP-TFA model), instead of providing the necessary resources to attract and retain experienced professionals to educate our most at-risk students. Silver bullets are simple, cheap and easy, but they always fail. Amazingly, even Texas is ahead of the administration on this one. It's time to ignor the phony "reformers" and get down to the task of improving education in America. The task will be slow, difficult and expensive and the more money wasted on NCLB, Race to the Top, TFA, NTP and the like will only make it harder, longer and more costly to undue the damage they have done.

Posted by: mcstowy | June 8, 2010 1:17 PM | Report abuse

If teachers are rewarded by how well students do on standardized test scores (a known false measure of student progress) some teachers will no longer be willing to teach students who are naturally slow, or students who never do homework, or students who don't have stable homelives (and thus perform poorer in school.)

I don't want to see teachers trying to influence principals to get higher performing students in order to make more money. But that is what is certain to happen.

Posted by: aby1 | June 8, 2010 1:18 PM | Report abuse

Happy Teacher, I’m surprised that you would comment on what you see in others as a “clear pattern of rushing to judgment” when just yesterday on Jay Mathews blog,* you said:

“It is safe to say that the state of education has declined in recent decades, and from what I have seen, it is quite related to teachers treating students like they need to be felt sorry for, instead of holding them to the task at hand: learning.”

However, when I asked you to cite your sources, you replied, “I don't have studies to cite, I am just basing that on my experience in the classroom.” Then you went on to say that you “would imagine” some studies would “touch on this.”

*
http://voices.washingtonpost.com/class-struggle/2010/06/dc_contract_gives_new_teacher.html

Posted by: efavorite | June 8, 2010 2:46 PM | Report abuse

Principals will start taking money from teachers to get the higher performing students. Anyone who does not realize this is going to happen does not understand human behavior very well.

Posted by: aby1 | June 8, 2010 4:06 PM | Report abuse

Below are a few more random quotes from the 99 page Colorado study cited above. I would certainly not characterize this as a study that indicates that “pay for performance” works.

“Teachers who have voluntarily opted into the ProComp program slightly outperform their non-participant colleagues, though differences are less pronounced when adjusted for individual differences between teachers who choose to participate and those who do not. Mixed effects models offer little evidence of significant changes in effects for voluntary participants after implementation compared to the effects before implementation”

“Teachers’ percentages of high-growth students (students with student growth percentiles above the 55th percentile) had little relation to teachers’ reported changes in their instructional behaviors”

“This study is observational (rather than experimental) in nature. Where possible we’ve employed quasi-experimental techniques toward the goal of understanding ProComp’s causal effects. That said, interpretation of results must be carried out with consideration of potential study confounds and limitations”

“Finally, achievement analyses reported herein are necessarily constrained to teachers of students who complete CSAP mathematics and reading assessments. This is relatively small subset of teachers relative to the entire ProComp‐eligible teacher pool. Any inferences regarding ProComp and achievement, then, must be limited to effects on teachers characterized by this subset.”

Posted by: efavorite | June 8, 2010 4:08 PM | Report abuse

Yes efave...I made the cardinal sin of discussing my opinion and observations on a blog. And no one has done any research on my opinion to back me up, so I lose the right to have an opinion?

I also stick by my assertion that there is a rush to evaluate the "effectiveness" of the performance pay programs, though I cannot find a study on this opinion either. My apologies.

Posted by: HappyTeacher | June 8, 2010 10:28 PM | Report abuse

We all have the right to express our opinions. There's no sin in that.

I think I know enough of logic to say consistency is also nice and will suffice.

Posted by: efavorite | June 8, 2010 10:52 PM | Report abuse

I see nothing inconsistent about making observations and expressing opinions on two different topics.

Sorry, I couldn't think of a word that rhymed with topics.

Posted by: HappyTeacher | June 8, 2010 11:55 PM | Report abuse

Good morning, Happy Teacher – the inconsistency I noted was in your apparent change in attitude about making definitive statements.

As for the rhyming, it’s not original; I borrowed from Robert Frost.

Posted by: efavorite | June 9, 2010 7:57 AM | Report abuse

We've had over 200 years to design an effective public education system and we still haven't gotten it ‎right, but the reform movement gets only 2 years to get a well designed performance pay system? I ‎don't think we reformers are rushing to judgment. What we are trying to do is study the system to ‎allow for well reasoned judgment. The only way to study education reforms is to implement them, ‎give them some time for any impact to occur, measure the impact, tweak the reform, give the tweak ‎time to develop, and measure again. Only after all this can solid judgment take place. Like many ‎posters stated above, education is a complex issue. It would be a miracle if noticeable results from any ‎education reform occurred in two years or even 10.‎

Posted by: peabody2671 | June 11, 2010 10:49 AM | Report abuse

The comments to this entry are closed.

 
 
RSS Feed
Subscribe to The Post

© 2010 The Washington Post Company