Network News

X My Profile
View More Activity


Posted at 4:30 PM ET, 07/13/2010

Rhee’s problem with D.C.’s new test scores

By Valerie Strauss

D.C. Schools Chancellor Michelle Rhee has a problem, and it’s not the fact that elementary school standardized test scores just went down (at a bad time for Mayor Adrian Fenty, who appointed Rhee and is seeking reelection).

The problem is that she has made rising standardized test scores a central measure for achievement -- hers, students and teachers.

When test scores go up, as they very often do when a great emphasis is put on the results and teachers “teach to the test” (either consciously or subconsciously), it is easy to claim credit. School reforms are working! Yeah!

But scores invariably go down after a time, no matter who is giving them and who is taking them, and they do so for reasons that may have nothing to do with the teacher, or the student, or the schools district chief.

My colleague Bill Turque reported Tuesday that reading and math test scores declined somewhat this year in D.C. elementary schools, halting a two-year run of significant gains and dealing a setback to Rhee. Middle and high schools showed continued gains in reading and math on the District of Columbia Comprehensive Assessment System (DC CAS), administered every April. The decline, it should be said, was much less than the overall improvement in the past few years.

But a rise in scores doesn’t necessarily mean that more student learning took place, and a decline doesn’t necessarily mean that students learned less.

Researchers in the field say that it is the nature of standardized tests that they rise from year to year when the same design of a test is given in the same schools. And they go down when a new design is given, or when a different demographic of students takes the test, or a bunch of kids in a class had a cold, or... well, you get the idea.

There are too many variables that can affect the scores of a single test to make the result completely reliable, which is why scores should never be used as a sole measure for any high-stakes decision.

“Test scores going down can reflect less adequate teaching and learning but it may reflect so many other things that you can’t be sure,” said Robert Schaeffer, public education director of the nonprofit National Center for Fair & Open Testing, or FairTest. “You can’t look at one-year test scores changes.”

So, feel free to judge Rhee on her insistence on placing so much importance on test scores (as have today’s other titans of education reform, such as Joel Klein in New York City). She believes they are so important, in fact, that she recently announced that she wants to expand their use in city schools, so that, in time, every D.C. student from kindergarten through high school is regularly assessed to measure academic progress and teacher effectiveness.

But judging her reforms on the actual test scores, well, as my kids say, “Not so much.”

Rhee said she and her team would “dig into the data” to find out why the elementary reading proficiency rate, which had risen 11 percentage points from 2007 to 2009, fell 4.4 points, to 44.4 percent, and why, after rising 20 percentage points from 2007 to 2009, the elementary math proficiency rate dipped 4.6 points this year, to 43.4.

The proficiency rate is essentially a measure of the portion of students who pass the tests.

This is just one reason why the current fashion in school reform to link teacher pay to standardized test scores is so wrong. So while Rhee digs into the data, it might do her well to dig into the reasons she thinks standardized test scores are worthy of being used for high-stakes decisions.

My colleague Jay Mathews, the longtime dean of education reporters in the country who for years championed the use of test scores to rate schools, suddenly just had an epiphany. He wrote in this recent post on his Class Struggle blog:

"I have to question my own judgment and fairmindedness when I ignore--for three years!-- a report that raises important questions about the way we have been using test scores to rate schools.

"I have always been open to better ways of assessing how our children are taught. But I usually say standardized tests are the best available tool at the moment. So I am embarrassed that it took me so long to read “Keeping Accountability Systems Accountable” by Martha Foote, published in the Phi Delta Kappan education journal in January 2007.

"I am indebted to the Monty Neill, executive director of Fairtest: The National Center for Fair and Open Testing, for pointing me toward the article and its author. Foote is director of research for the New York Performance Standards Consortium, which she describes in her article as “a coalition of 28 small, diverse public high schools across New York State that exemplify education reform based on strong commitment to school-as-community, to ongoing professional development, and to innovative curricula and teaching strategies.” That sounds good to me, but it gets better."

Read the rest here, at Jay’s blog. I sure wish Michelle Rhee would.

Follow my blog all day, every day by bookmarking washingtonpost.com/answersheet. And for admissions advice, college news and links to campus papers, please check out our Higher Education page at washingtonpost.com/higher-ed Bookmark it!

By Valerie Strauss  | July 13, 2010; 4:30 PM ET
Categories:  D.C. Schools, Standardized Tests  | Tags:  CAS scores in d.c., d.c. CAS, d.c. schools, d.c. standardized tests, d.c. test scores down, d.c. test scores mixed, d.c. test scores rise, fairtest, michelle rhee and reform, monty neill, rhee and test scores, tmichscho  
Save & Share:  Send E-mail   Facebook   Twitter   Digg   Yahoo Buzz   Del.icio.us   StumbleUpon   Technorati   Google Buzz   Previous: Florida standardized test results challenged
Next: How we evaluate relief pitchers and teachers

Comments

Will the Washington Post stop reporting these bogus results?

2009 national 4th grade tests DC proficient or above
math 17 percent
reading 11 percent

Now we have claims of DC tests results of elementary schools that indicate 44 percent of students being proficient in reading and 43 percent being proficient in math.

Bogus tests of Ms. Rhee for 2010
proficient or above
math 43 percent
reading 44 percent

Yes DC can degrade tests to such an extent that it can make false claims but unfortunately for Ms. Rhee, DC has so many Title 1 poverty public schools that she can not use bogus test results when national tests for DC clearly indicates significantly lower numbers of students that are proficient.

By the way the 2009 national tests for DC of proficient and above in the 8th grade were:
math 11 percent
reading 14 percent

Ms. Rhee has consistently presented test results that are bogus when viewed in the context of test results of national tests for DC as her proof of improvement in the school system.

The Washington Post and the supporters have to stop making claims that patently bogus test results have shown the effectiveness of Ms. Rhee.

The only effectiveness Ms. Rhee has shown so far is the effective use of bogus test results to hoodwink the Washington Post.

http://nces.ed.gov/nationsreportcard/states/

Posted by: bsallamack | July 13, 2010 4:58 PM | Report abuse

"But I usually say standardized tests are the best available tool at the moment."

Jay Matthews said that? Does he know anything about assessment? Or did he have an epiphany when the elementary scores went down?

Posted by: aby1 | July 13, 2010 5:18 PM | Report abuse

Everyone talks about standardized tests.

Ms. Rhee has shown us with her bogus tests where proficient on her tests is equivalent in many cases to failing basic skills on national tests, that in many case the only standards in standardized tests can be that the tests require paper and pencils.

Posted by: bsallamack | July 13, 2010 5:56 PM | Report abuse

Rhee said she and her team would “dig into the data” to find out why the elementary reading proficiency rate, which had risen 11 percentage points from 2007 to 2009, fell 4.4 points, to 44.4 percent, and why, after rising 20 percentage points from 2007 to 2009, the elementary math proficiency rate dipped 4.6 points this year, to 43.4.
....................................
DC tests should show dramatic improvement. There is no concern that proficient on a DC test is equivalent to failing basic on a national test.

The team will "dig into the data" so that questions that were answered incorrectly can be removed from future tests.

It is obscene that the Washington Post gives any credence to these bogus tests and their bogus results.

Posted by: bsallamack | July 13, 2010 6:05 PM | Report abuse

But a rise in scores doesn’t necessarily mean that more student learning took place, and a decline doesn’t necessarily mean that students learned less.
...............................
With the bogus tests of DC that equate proficient in many cases with failing basic on national tests it would be more appropriate to write:

A rise in scores on a bogus test does not mean that more student learning took place, and a decline on a bogus test does not mean that students learned less.

Bogus tests in fact mean nothing except the gullibility of the Washington Post and their failure to question obviously bogus tests.

One usually has respect for Ms. Strauss but she is stretching her credibility in her comments about the tests of DC that are obviously bogus as proven by a comparison to the national tests results of 2009 of DC public schools.

Posted by: bsallamack | July 13, 2010 6:16 PM | Report abuse

Story for the New York Times.

Ms. Rhee in the past has made claims of significant gains in public education based upon DC test results.

In the past there never has been totally objective evidence to refute these claims or support these claims.

Recent DC test results released by Ms. Rhee show that these claims were not valid.

DC test results for 2010 for elementary students
proficient or above
math 43 percent
reading 44 percent

These results are impossible to reconcile with the national tests.
2009 national 4th grade tests DC
proficient or above
math 17 percent
reading 11 percent

It is impossible to make any valid claim regarding DC test results when the concept of proficiency on a DC test is so far removed from the concept of proficiency on national tests.

It is apparent that improved test scores in DC have been simply obtained by lowering the standards of these tests and that there is no evidence to support the claims of Ms. Rhee of improvement of public education in DC.

http://nces.ed.gov/nationsreportcard/states/

Emailed to the public editor and reporters covering education.

Posted by: bsallamack | July 13, 2010 7:05 PM | Report abuse

aby1 says: "Jay Matthews said that? Does he know anything about assessment? Or did he have an epiphany when the elementary scores went down?"

Exactly. Mathews also reported that he had no use for the achievement gap when it was learned that the gap had increased in under Rhee.

http://voices.washingtonpost.com/class-struggle/2009/12/why_i_have_no_use_for_the_achi.html

Posted by: efavorite | July 13, 2010 9:02 PM | Report abuse

Scores went up in the charter schools.

Posted by: parentof2redheads | July 13, 2010 9:05 PM | Report abuse

Well. If standardized test is useless, I guess the NAEP is useless too.

I can agree on that there are so many factors that can affect the test score. However, any judgment do that. Even the teacher is not immune from flaw.

A good test take a lot of design efforts and tests. This is why we need ACT/ETS/SAT .. or US DE for NAEP. The bashing of standardized test is wrong. The article just showed the other side of the story - you can't teach to the test - especially a well designed one. Otherwise, how could the score fell?

It's not that standardized test is useless. It simply that people need to put thoughts into it and this apply to all kind of tests. Even an essay test is not without flaw. For example, the score could be totally depend on who grades the essay.

Not all kind of tests can test every thing.


Posted by: NeverRead | July 14, 2010 8:42 AM | Report abuse

The bashing of standardized is correct. You obviously don't know much about them.

Posted by: jlp19 | July 14, 2010 3:25 PM | Report abuse

parentof2redheads,

Test scores went up for high school. What I am afraid is that if charter schools are allowed to drop non-performing students they are going to have a higher percentage of higher testors.

Posted by: jlp19 | July 14, 2010 3:28 PM | Report abuse

It's not that standardized test is useless. It simply that people need to put thoughts into it and this apply to all kind of tests.

Posted by: NeverRead
................................
People like Ms. Rhee do put thought into these test. It takes effort to lower the standards on a test so that one can claim that a student is proficient when actually the student is not proficient.

Every years Ms. Rhee claims really significant gains on bogus DC tests while the national tests that are given every two year show no signs of significant gains for DC.

Posted by: bsallamack | July 14, 2010 6:05 PM | Report abuse

An idea for Ms. Rhee.

Apparently Ms. Rhee can no longer count on lowering the standards of the DC tests to count on more claims of improvement in the Title 1 poverty public schools.

Start a large 50 percent of salary bonus program for teachers and principals for improved test scores.

Then the school system will just have to make sure principals hand out erasures for the teachers at test time and provide sufficient time for teachers in private after the tests are handed in.

Posted by: bsallamack | July 14, 2010 6:07 PM | Report abuse

The chancellor's problem is that tests are all she's got--her signature achievement as a teacher in Baltimore "scaled up" to an entire public school district in the course of which, teacher compensation/WTU had to be "re-formed" so she is free to franticly hire and fire in a desparate search for people who can repeat her "achievement." It's completely backward and empty. If the environment in the schools and the system as a whole is not conducive to teaching and learning CONTENT for the next level of learning and life, then what is there TO test? Testing has its place in the process but to base everything on it is ridiculous!

Posted by: 1citizen | July 15, 2010 3:22 PM | Report abuse

The comments to this entry are closed.

 
 
RSS Feed
Subscribe to The Post

© 2010 The Washington Post Company