Buster Posey and the value-added teacher debate
A new report on the controversy over rating teachers by how much their students' scores improve asks, sort of, this odd question: Should the San Francisco Giants keep rookie of the year Buster Posey on their team next year?
"What kind of a question is that!!??" Giant fans say. I used to be a Giants fan, but it's hard when my toddler grandson Ben, a Los Angeles resident, yells "Booooo!" every time that team is mentioned. Nonetheless, trading Posey would seem idiotic even to him.
But the Brown Center on Education Policy at the Brookings Institution, in its new report "Evaluating Teachers: The Important Role of Value-Added," suggests that Posey's admirable .305 batting average, in the view of those who warn against using valued-added data in judging teachers, does not seem so great.
It notes that "the correlation in test-based measures of teaching effectiveness between one school year and the next lies between .20 and .60 across multiple studies, with most estimates lying between .30 and .40. A measure that has a correlation of .35 from one year to the next produces seemingly troubling statistics in line with our conceptual discussion of classification errors."
Critics of value-added assessments have noted that only about a third of teachers ranked in the top quartile of value-added based on one academic year's performance would appear in the top quartile the next. Having set us up to think that .35 is not such a great correlation, the Brown Center paper's six authors (only one of whom, Susanna Loeb of Stanford, seems to live in Giants territory) slyly dump this on us:
“The between-season correlation in batting averages for professional baseball players is .36. [They cite a 2000 paper in The American Statistician, "Do baseball players regress to the mean?", which of course we all remember.] Ask any manager of a baseball team whether he considers a player's batting average from the previous year in decisions about the present year."
That shows how nasty this argument over value-added assessment has become. The L.A. Times created a furor when it published the value-added records of thousand of local teachers. School districts across the country are battling over the issue. The Brown Center paper pushes a stick into this red ant nest by noting that the critics of value-added don't mention how often we use other metrics with similar reliability in big decisions.
Picking college applicants based on SAT scores, selecting hospitals based on mortality rates, recruiting realtors based on home sales volume -- all involve correlations similar to those for picking a teacher based how much her students' achievement rates improve. The authors warn against setting "unrealistic expectations for the reliability or stability of value-added. Value-added evaluations are as reliable as those used for high-stakes decisions in many other fields."
If your goal is raising student test achievement, the paper says, "value-added is superior to other existing methods of classifying teachers."
It is a clever contribution to the debate, certain to make you howl or cheer, depending on your point of view. It will not, however, increase in any way the infinitesimal chance that the Giants will ever trade Posey to my grandson's beloved Dodgers.
| November 17, 2010; 9:00 AM ET
Categories: Jay on the Web | Tags: Buster Posey, batting averages have similar flaws, value-added criticized for mediocre reliability as a measure of teacher effectiveness year to year, value-added teacher assessment
Save & Share: Previous: Veterans, struggling students need more college support
Next: Hiding exams from students
Posted by: lisamc31 | November 17, 2010 9:32 AM | Report abuse
Posted by: Jay Mathews | November 17, 2010 11:13 AM | Report abuse
Posted by: JackS2 | November 17, 2010 12:24 PM | Report abuse
Posted by: Wyrm1 | November 17, 2010 12:49 PM | Report abuse
Posted by: fairfaxvaguy | November 17, 2010 1:31 PM | Report abuse
Posted by: Wyrm1 | November 17, 2010 2:19 PM | Report abuse
Posted by: lisamc31 | November 17, 2010 2:29 PM | Report abuse
Posted by: Title1SoccerMom | November 17, 2010 2:30 PM | Report abuse
Posted by: Jay Mathews | November 17, 2010 2:55 PM | Report abuse
Posted by: mmccabe4724 | November 17, 2010 7:28 PM | Report abuse
Posted by: staticvars | November 18, 2010 1:01 AM | Report abuse
Posted by: davismiller123 | November 18, 2010 4:28 AM | Report abuse
Posted by: ceolaf3 | November 18, 2010 7:39 PM | Report abuse
Posted by: jeffcherniss | November 21, 2010 2:40 AM | Report abuse