Network News

X My Profile
View More Activity


Posted at 6:30 AM ET, 07/14/2010

How we evaluate relief pitchers and teachers

By Valerie Strauss

My guest is Robert Pondiscio, director of communications at the Core Knowledge Foundation who launched the Core Knowledge Blog, where this piece was first published.

By Robert Pondiscio
Over on Twitter, my friend Stephanie Germeraad, who is nearly as passionate about sports as she is about education, suggests schools ought to steal a page from baseball when it comes to teacher seniority.

Commenting on the decline of legendary closer Trevor Hoffman, she tweets a quote from Alan J. Borsuk: “Schools can learn from baseball. Brewers wouldn’t start Hoffman just because he’s been pitching longer.” The point is that seniority is no guarantee of quality.

Fair enough. But here’s a sobering truth: We are far more capable of measuring the effectiveness of relief pitchers like Hoffman than classroom teachers.

If you’re a casual baseball fan, you might know a few ”facts” about the pitchers on your favorite team: their won-loss record, their ERA (the number of “earned runs” allowed per nine innings), or their WHIP (walks and hits per innings pitched).

To an expert, such statistics scratch the surface at best, and may even be irrelevant. Wins are a function of a team’s offense, for example, as much as a pitcher’s effectiveness, while ERA and WHIP are strongly influenced by the defensive ability of the other eight men on the field. An outfielder with greater range for example, will record an out on a ball that a lesser defender lets fall for a hit. Same pitch, same swing, different outcome.

Among baseball geeks, you often hear discussions of fielding independent pitching, or ”FIP,” a measure of the things a pitcher is directly responsible for such a strikeouts, home runs and walks. FIP helps you understand how well a pitcher pitched, regardless of how well the team played behind him.

Data even helps teams decide what kind of pitchers are best suited to their stadiums through analysis of “park effects.” A fly ball pitcher (yes, they keep track of fly balls, line drives and ground balls hit off every pitcher) might prosper in a big stadium like New York’s Citi Field, but allow lots of home runs in a bandbox like Philadelphia’s Citizens Bank Park.

A pitcher who “pitches to contact” (i.e., doesn’t strike out a lot of hitters) is fine if your team’s defense is strong. If not, you might spend more to sign pitchers who are strikeout artists. Data even helps spot problems as they occur. Fans of the New York Mets are concerned that all-star pitcher Johan Santana’s fastball is topping out below 90 miles an hour of late, making his changeup, a slow-speed pitch, less likely to fool hitters expecting the fastball.

To a baseball fan statistics are a revelation. The granularity and specificity are illuminating.

You can see, if you’re so inclined, a pitcher’s FIP, ERA, strikeouts, and his strikeout-to-walk ratio. The percentage of batted balls that were hit on the ground, in the air, or for line drives can speak volumes about a pitcher’s effectiveness. When a player’s agent goes to negotiate his contract, he can even discuss his “Wins Above Replacement” (WAR), a statistic that measures the total value of a player over a given season compared to an average replacement player.

If these kinds of numbers thrill you, adding depth and nuance to your love of baseball, thank Bill James. It is no overstatement to say that no one has had a greater impact on baseball in the last 25 years than James, who pioneered and named the field of sabermetrics, the use of detailed statistics to analyze baseball team and player performance.

James has made a career of demonstrating the factors that lead to teams scoring runs and winning games, and how the efforts of individual players contribute to wins. Some of his insights have been legendary and have overthrown time-honored beliefs about the game–why RBIs matter less than on-base percentage, for example. Or why stolen base attempts tend to hurt a team’s offense.

Before Bill James, baseball was all batting averages, bromides and intangibles–a century of baseball men who knew what they knew based on experience, instinct and rudimentary data.

We are in the test scores, bromides and intangibles era of measuring teacher quality. If you’re a prinicipal, wouldn’t you love to know the “school effects” of teacher performance when it came time to make hiring decisions? Would it change your perception of merit pay if there was a classroom equivalent of FIP–the factors directly under a teacher’s control?

What if we could compensate teachers based on their replacement value compared to an average first year teacher?

“It’s far more than win/loss/ERA/WHIP” is the clubhouse mantra,” Stephanie tweeted, defending her assertion that education can profit from baseball’s example. ”Difference is, baseball doesn’t say they therefore can’t do it,” she wrote.

Not quite right. In baseball there is data–lots of it–to measure effectiveness clearly and fairly. Difference is ”it’s far more than test scores” is not a mantra in ed reform.

Education awaits its Bill James.

-0-

Follow my blog all day, every day by bookmarking washingtonpost.com/answersheet. And for admissions advice, college news and links to campus papers, please check out our Higher Education page at washingtonpost.com/higher-ed Bookmark it!

By Valerie Strauss  | July 14, 2010; 6:30 AM ET
Categories:  Guest Bloggers, Teachers  | Tags:  baseball facts, bill james, evaluating baseball players, evaluating teachers, fplay, johan santana, teachers  
Save & Share:  Send E-mail   Facebook   Twitter   Digg   Yahoo Buzz   Del.icio.us   StumbleUpon   Technorati   Google Buzz   Previous: Rhee’s problem with D.C.’s new test scores
Next: Cool Summer Class: 'Boot camp' revisited

Comments

Robert,

This was an excellent earlier post this year on your blog.

"Difference is, baseball doesn’t say they therefore can’t do it." But baseball does say they can't do it. If a general manager examines stats the way James does he can determine a player's legitimate worth. If the player has demonstrated they can do their job, the GM will be inclined to hang onto that player. If, however, the data indicates the player has deficiencies, the GM can either waive the player, trade him, or simply not re-sign him.

Posted by: phoss1 | July 14, 2010 7:37 AM | Report abuse

Ouch. This illogical analysis just makes the brain hurt. Aside from the basic premise that in baseball one can be washed up by the age of 35 or so and baseball is a highly physical engagement whereas in education, the widsom of a teacher may still be rising at the age of 35 or so and whose profession utilizes (should) "high order thinking skills" all day. Plus, if a teacher is in a lull, he may not need to be transferred, non-renewed, whatever, he may tho' need workshops, classes, a more enlightened principal to pinpoint areas of improvement and provide assistance, peer support and guidance, whatever. As W.E. Deming would have said (paraphrased using somewhat faded memory of the statement), did you hire them that way, or did you kill them?

Posted by: shadwell1 | July 14, 2010 9:52 AM | Report abuse

This is based on an article published in the New Yorker several months ago. I agree with Shadwell.

Posted by: lacy4 | July 14, 2010 10:44 AM | Report abuse

@shadwell I agree with you, sort of. I'm not arguing that we should evaluate teachers the way we evaluate baseball players. I'm merely pointing out that the detailed, nuanced data we have about baseball makes player evaluations fair. There's nothing like it when it comes to teachers. We're reaching vast conclusions about teachers with half-vast data.

Posted by: rpondiscio | July 14, 2010 10:48 AM | Report abuse

Testing is a "necessary evil". Why are so many people up in arms about testing in schools as a measure of success or lack thereof, but will not apply the same argument to the SAT exam when it's time for college.

If I score low on the SAT, can I not succeed at Harvard?

Posted by: rickyroge | July 14, 2010 10:52 AM | Report abuse

Can we stop with this nonsense about evaluating teachers instead of buying into the kool aid of the politicians.

Most public schools in the country do a good job in public education. The majority of public schools in this nation are middle class and affluent public schools that turn out large numbers of students for colleges. Most parents are satisfied with these public schools.

The disaster Title 1 poverty public schools thank God do not represent the public schools in this nation.

Imagine if national tests indicated 56 percent of students in one of these middle class or affluent public schools systems failed 4th grade reading.

All of the public leaders of the community would be forced to resign. The head of the school system and all the principals at the elementary schools would be replaced. Parents who pay high property taxes would be outraged and action would be swift. Yes teachers would be replaced but not until all the public leaders and school administrators were forced out and replaced.

Stop the nonsense and the attempt by the politicians to taint the public schools in this nation with the disaster of Title 1 poverty public schools.

Most public schools in the nation do an excellent job in evaluating teachers without expensive and doubtful standardized tests and expensive computer systems.

Let the politicians start programs specifically aimed at the Title 1 poverty public schools and stop trying to force expensive and questionable teacher evaluation methods on the majority of public schools that do not need them. Have the politicians and columnist stop tainting public school teachers as lazy bums that have to be made to do their job. Teachers in most schools do a good job everyday and there is no need for performance pay.

The politicians have to stop the pretense that there is problem in public schools in this nation. The Title 1 poverty public schools have always been a disaster but they do not represent the vast majority of public schools and teachers in this nation.

The real problem to the majority of public schools in this nation is the politicians.

Posted by: bsallamack | July 14, 2010 12:46 PM | Report abuse

How to evaluate the pitchers of the last place Washington DC Title 1 baseball team.

Have the baseball manager of the team set up standardized tests to evaluate the pitchers.

Pitchers that can roll a baseball over the plate are considered proficient.

Have the baseball manager convince the press that there is real improvement of the pitchers based on these standardized tests of pitchers.

Attendance at games of the Washington DC Title 1 team goes up even though the team is still in last place.

The beer drinking fans cry out that every baseball team in America should follow suit with improving their pitchers, even though most of these teams have excellent pitchers.

Posted by: bsallamack | July 14, 2010 1:06 PM | Report abuse

Testing is a "necessary evil". Why are so many people up in arms about testing in schools as a measure of success or lack thereof, but will not apply the same argument to the SAT exam when it's time for college.

If I score low on the SAT, can I not succeed at Harvard?

Posted by: rickyroge
..................................
There already is enough testing in the majority of public schools that are not Title 1 public poverty schools.

The majority of public schools in this country that are not Title 1 poverty public schools have been using testing and evaluation of teachers methods for years. There would be outrage across the country if these schools were performing as badly as Title 1 poverty public schools. Parents in these schools would be up in arms.

The majority of public schools do well and do not need new expensive tests and computer systems.

Public schools system that turn out large number of graduates need money for the effective teachers that the schools have been forced to let go because of the budget crisis of the states and not unnecessary expensive tests.

Yes there has always been a problem with the disaster Title 1 poverty public schools but it makes no sense to adopt expensive and questionable methods for the majority of public schools where there is no problem.

Let us stop this nonsense about expensive tests and computer system while the public schools are having difficulty finding the money to pay for teachers.

And let us stop this nonsense about ineffective and lazy teachers. Unlike the disaster Title 1 poverty public schools that are subsided by the government most public schools in the country are paid for by taxpayers through property taxes and these taxpayers would not tolerate if their public school systems were riddled with inefficient and lazy teachers.

Posted by: bsallamack | July 14, 2010 1:41 PM | Report abuse

Right on Bsallamack...

I think our closest thing to Bill James may have been Jerry Bracey, (may he rest in peace). But of course, he took a much broader view. I think he too may have agreed with Bsallamack.

While I appreciate that this analogy is in defense of teachers, I nonetheless think it is a bad one. You are comparing the evaluation of a pitcher, dependent almost entirely on physical prowess to the evaluation of an occupation based almost entirely on intellect and interpersonal skills. Let us not forget that an outstanding pitcher is a very rare individual... (I'm thinking here of Tim Lincecum... go SF Giants!) who is performing a repetitive move with slight, but precise variations while simultaneously destroying his rotator cuff and God knows what else in the process.

Not only are we far more capable of evaluating a pitcher, it is also far easier to do so than evaluating a teacher. Plus you have literally millions of people watching that player in game after game who can support the numerical data.

I get what you are saying... there are far more nuanced statistics for evaluating pitchers than teachers. And standardized test scores are a very rough statistic. But I also think that trying to reduce the interpersonal, intellectual, and emotional work of a teacher to mere numbers is questionable, if not impossible, and will always yield less than accurate results.

Posted by: Incidentally | July 14, 2010 1:46 PM | Report abuse

Sports analogies vis-a-vis teaching is not just apples and oranges, it's not remotely in the same food group!!!!

When contemplating an analysis of effective teaching, it would probably be helpful to look at other places where teaching takes place, and the basic modes of teaching.

To take the modes first: setting an example, lecturing to,behavior management, demonstrating various techniques of various tools, inspirational guiding, interventions, etc.
This is a long and complex list, and takes much intellect, training, practice and determination to master, AS DOES EVALUATION
of these various endeavors.

Where does teaching take place? Almost anywhere.....
The greatest teachers arguably are from the great philosophers,religious leaders and activists in the world: Socrates, Ghandi, Jesus Christ, Mother Teresa, and the Martin Luthers come to mind. How would we choose to evaluate them? Time....
they have stood the tests of time. How can we possibly evluate our school teachers just in the context of a single year when many influence students' lives way beyond their school careers.
On the less-than-world spiritual leader examples of teaching,outside of school there are many 'teachers' to evaluate: the
media darlings (soap stars, rock stars, movie stars,etc.),business enterprises (fashion, liquor stores, cars, gyms,banks, etc.) what are they sellings - oops, I meant teaching - community groups, from the meaningful (PTA,Scouts) to the dangerous (gangs)...... . . . . and,most importantly, PARENTS.


All of the above have influence, and in that sense, teach our children.

A note on teacher evaluation as it exists today: It is very irritating to read so many comments from people, particularly politicians that assume teachers are not, and have not been, evaluated. Any teacher who went through a TRUE TEACHING PROGRAM FROM A RESPECTED UNIVERSITY was evaluted every step of the way, from scrutiny of papers written, tests taken, field observations, student teaching, Graduate admissions (GREs,Millaer's Analogy)to supervisory and principal evaluations once hired. To have someone like Michelle Rhee be in a position of revamping teacher evaluations is so insulting, it defies description.

From Crosby, Stills & Nash ( did I leave out a fourth?): "Teach Our Children Well"......

Posted by: PLMichaelsArtist-at-Large | July 14, 2010 1:57 PM | Report abuse

For Valerie Strauss

We need columnists telling Americans the real story.

We need columnists telling Americans that the disaster of Title 1 poverty public schools, like the Title 1 poverty public schools of Washington DC do not represent the overwhelming majority of public schools in America.

We need columnists to tell Americans that the majority of public schools that do a good job in public education are being forced to let go of effective teachers because of lacks of fund.

We need columnists to tell Americans that in the majority of public schools there is no need for expensive tests and computer systems to evaluate tests since these schools have always been effective in evaluating teachers.

We need columnists to tell Americans the real story about the majority of public schools in America where ineffective and/or lazy teachers are quickly let go and there are dozens of qualified applicants to teach in these schools.

We need columnists to tell Americans to stop listening to the politicians who are not offering any new ideas regarding the disaster of Title 1 poverty public schools, and are only trying to divert the public's attention from their ineffectiveness to deal with the disaster of these schools by trying to pretend that they reflect public education in the majority of public school in this country.

We need columnist to tell Americans the truth about public education in this nation.

Posted by: bsallamack | July 14, 2010 2:03 PM | Report abuse

@incidentally As a former teacher, I agree that "trying to reduce the interpersonal, intellectual, and emotional work of a teacher to mere numbers is questionable." That's why the onus is on those who insist that it's possible to demonstrate exactly how it can be done.

Posted by: rpondiscio | July 14, 2010 3:04 PM | Report abuse

If I score low on the SAT, can I not succeed at Harvard?

Posted by: rickyroge

Of course you can do well at Harvard if you score low on the SAT. It just depends on your efforts.

Posted by: jlp19 | July 14, 2010 3:04 PM | Report abuse

"We need columnists telling Americans that the disaster of Title 1 poverty public schools, like the Title 1 poverty public schools of Washington DC do not represent the overwhelming majority of public schools in America."

AMEN!

Posted by: jlp19 | July 14, 2010 3:09 PM | Report abuse

Here is the story on the nonsense of standardized tests and teacher evaluations from the Title 1 poverty public schools of Washington DC.

National tests for DC 2009
Proficient or above
math 4th grade 17 percent
math 8th grade 11 percent
reading 4th grade 17 percent
reading 4th grade 14 percent

Now DC is claiming for their tests of 2010 that 43 percent are proficient in math and 44 percent are proficient in reading.

The DC scores of proficiency are apparently bogus as they are totally different from the national tests. The tests have apparently been degraded where scoring proficient is totally meaningless.

The DC school system is calling for more money for standardized tests since it allows for the pretense of claims that there has been improvement in their disaster Title 1 poverty public schools.

For the cost of standardized testing in DC for one year, DC can now claim that in one year the number of students scoring proficient are double, and three times higher than the scores on the national tests of 2009.

Contrary to the politicians, expensive standardized testing and evaluations will not raise performance in the disaster Title 1 poverty public schools.

The only thing that could be said about the tests in DC is that if America allowed the politicians and fakers in educator to impose their poor ideas of standardized testing and evaluation on the meat we eat, we would all be eating roadkill.

Posted by: bsallamack | July 14, 2010 3:32 PM | Report abuse

In fact, ERA and WHIP reflect mainly the pitcher's performance. ERA, for example, doesn't take into consideration unearned runs (runs caused by defensive blunders by one of the other eight men on the field). WHIP is much the same and the other players have only a small effect.

I am a teacher (though I don't like the term very much anymore as what we face is not really understood by all of the chatterboxes and arm-chair quarterbacks. In addition, the field is not much respected compared to those such as architects and lawyers, though from what I can tell the complexity and difficulty level is similar.

Speaking of quarterbacks, the position offers a much more vivid analogy (explained more eloquently by Malcolm Gladwell of The New Yorker. We are responsible for moving the ball down the field with enormous resistance coming our way (administrative overreach, legal ramifications, poor facilities, uninterested or ill-equipped parents, etc, etc). We have to think on our feet, be agile and resourceful, and we need excellent protection from the offensive line or we're toast. That offensive line in a functional school is the principal and support staff, parents. The students are the running backs and receivers, the "skilled" players who score most of the touchdowns.

Posted by: thetensionmakesitwork | July 14, 2010 5:02 PM | Report abuse

Great piece -- and I am a big fan of juxtaposing sports and education to see what it can teach us. My own personal take on this issue, however, is that we both need the Bill James of education (see my column "What Would Theo Do?") AND we need to recognize that learning is more akin to soccer than it is to baseball (see "Dear Mr. President: Just Go With the Flow."). Both columns, btw, are accessible at http://www.huffingtonpost.com/sam-chaltain. I'll be curious to know if you find either piece to be a helpful addition to the points you raise here, and/or if you disagree with my premises.

Posted by: schaltain | July 16, 2010 11:53 AM | Report abuse

The comments to this entry are closed.

 
 
RSS Feed
Subscribe to The Post

© 2010 The Washington Post Company