Network News

X My Profile
View More Activity


Posted at 6:30 AM ET, 05/ 6/2010

Harvard study gives Race to Top winners bad grades on academic standards

By Valerie Strauss

One of the two states chosen by Education Secretary Arne Duncan as a winner in the first round of the $4 billion Race to the Top competition has academic standards that earned the grade of ‘F’ in a new study by Harvard University researchers, while the other state got a ‘C minus.’

The Education Next report by researchers Paul E. Peterson and Carlos Xabel Lastra-Anadón also shows that standards in most states remain far below the proficiency standard set by the National Assessment of Educational Progress. NAEP is known as the nation’s report card because it tests students across the country by the same measure and is considered the testing gold standard. States have their own individual student assessments designed to test students’ knowledge of state academic standards, which are all different.

This study, available on the Education Next website, comes on the heels of another analysis done by the Washington D.C.-based Economic Policy Institute, which concluded that the two first-round winning states, Tennessee and Delaware, were chosen through “arbitrary criteria” rather than through a rigorous scientific process.

Duncan chose the two winners from 16 finalists in the competition, which pits states against each other in a bid to win federal education funds by promising to institute school reforms favored by Duncan, such as expanding charter schools. Delaware won $100 million, or about $800 per student, and Tennessee was awarded $500 million, or about $500 per student. The second round is now under way.

In the Education Next study, the researchers looked at the percentage of fourth- and eighth-grade students in each state that performed proficient on the 2009 NAEP math and reading exams, and then compared that with the percentage of students deemed proficient on state assessments.

States with a similar percentage of students proficient on state tests and on the NAEP test earned an ‘A.’ But lower grades were given to states with high percentages of students proficient on state exams but with low percentages of proficient students on NAEP. The exact grade depended on how much lower state results were than the world-class NAEP.

Tennessee, with an 'F,' was at the bottom of the list of states and the District of Columbia. It reported that 90 percent of its fourth-grade students were proficient in math on Tennessee’s own assessment, though NAEP tests showed that only 28 percent were performing at the proficient level. Results in reading at the fourth- and eighth-grade reading levels were similar.

Delaware earned a ‘C minus.’ The state said that 77 percent of its fourth-grade students were proficient in math though on the NAEP only 36 percent were. In eighth-grade reading, Delaware said 81 percent of its students were proficient, though only 31 percent were proficient on NAEP.

Five states—Hawaii, Massachusetts, Missouri, New Mexico and Washington—earned an ‘A.’

Alabama and Nebraska joined Tennessee in earning an ‘F.’

The District of Columbia earned a ‘C,’ while Maryland and Virginia both got a D-plus.

The report will be published in the summer issue of Education Next, a scholarly journal published by the Hoover Institution and online by Harvard University.

It shows that only a very few states have changed their standards significantly since they first set them. Colorado raised its standards from a ‘D’ to a ‘B minus’ between 2003 and 2009, while South Carolina let its standards fall from ‘A’ level to a ‘C minus’, and Arizona standards fell from a ‘B minus’ to a ‘D’.

But changes in most states have been marginal. Tennessee has always received an ‘F’, while Delaware’s standards have fallen from a ‘C’ in 2003 to a ‘C-’ in 2005 and subsequently.

Under the federal No Child Left Behind law, states actually have an incentive to set low standards because that made it easier to meet the law’s requirements. States with especially high standards have had a harder time doing so. That peculiarity in the law is one of the things that Duncan has said he wants to remove when the Obama administration and Congress rewrite NCLB, probably next year.

Paul E. Peterson is the director of Harvard’s Program on Education Policy and Governance, where Carlos Xabel Lastra-Anadón is a Harvard Research Fellow.

-0-

Follow my blog all day, every day by bookmarking washingtonpost.com/answersheet And for admissions advice, college news and links to campus papers, please check out our new Higher Education page at washingtonpost.com/higher-ed Bookmark it!

By Valerie Strauss  | May 6, 2010; 6:30 AM ET
Categories:  Education Secretary Duncan, National Standards, No Child Left Behind, Race to the Top  | Tags:  Education Next, NAEP, National Assessment of Educational Progress, No Child Left Behind, Paul Peterson, Race to the Top, academic standards, education next, grading the states, state assessments, state content standards, state standards  
Save & Share:  Send E-mail   Facebook   Twitter   Digg   Yahoo Buzz   Del.icio.us   StumbleUpon   Technorati   Google Buzz   Previous: How U.S. school reform is linked to UK vote
Next: How Obama should set literacy goals

Comments

I find it interesting that DC got a C. I thought the standards in DC were strongly based on the Massachusetts standards, or at least that was Dr. Janey's plan. An interesting question would be how far did he fall short of his goal versus how much the city's standards have been diluted since Ms. Rhee took over.

Posted by: Mulch5 | May 6, 2010 7:50 AM | Report abuse

OUR INCREASING SCORE GAP

The DC-CAS scores (math & reading) in 2007 were, on average 20 points higher than the NAEP results. This was before the Chancellor's work took effect. In 2009, the DC-CAS scores (math & reading) were, on average 30 points higher than the NAEP results. This means that there was a 10 point rise in the SCORE GAP under Rhee between 2007 and 2009.

This year we do not have the NAEP to compare our DC-CAS results to, but most teachers ate reporting that many students are completing the test sections (40 mins each) in record times, and far fewer are unable to finish than last year. Additionally, the students are reporting that the test was far easier than last year.

In this election year, as the DC-CAS scores for 2010 come out in June, expect a parade for Fenty/Rhee. When our DCPS students "SHOCK THE WORLD" with the most impressive gains yet. And the WaPo editorial board will never consider this article, or the very real potential that the SCORE GAP might be increasing again next year when we will be able to compare our results with the NAEP.

Too bad ... the NAEP is not given in even numbered years, as we really could use "NAEP 2010" test results (won't exist) to expose a Chancellor willing and capable of manipulating any metric or budget to meet her immediate goals. Sad, but true.

Posted by: AGAAIA | May 6, 2010 8:23 AM | Report abuse

I don't know the details of the study, but from the article, it appears that there is a possible flaw in the way the states' grades are determined. In the high school where I teach, students who are randomly selected to take the NAEP test don't really have much motivation to put forth any effort. The test doesn't affect whether or not they graduate or move on to the next grade. For many students, it's meaningless. So, I can totally see states with higher passage rates on state exams than on the NAEP. From the point of view of the students, the NAEP is only important to a handful of invisible bureaucrats in DC.

Posted by: jgip_dr | May 6, 2010 8:51 AM | Report abuse

I want to share for this discussion an article that Guy Brandenburg has done looking at this gap in DC.

http://gfbrandenburg.wordpress.com/2010/04/15/another-credibility-gap-looks-like-rhee-reinoso-are-gaming-the-tests-to-try-to-look-good/

He shows that this gap has increased since Rhee took over.

Posted by: Mulch5 | May 6, 2010 9:12 AM | Report abuse

jgip_dr:

The DC-CAS does not affect a students grade or place a obstacle to graduation ...

A student could randomly fill out the exam and never be adversely affected in DCPS, but I understand that some private school admissions committees look at these results when considering students coming from DCPS.

However, a poor test performance will damage a student's teacher and school administration. So the adults are held accountable for students that fail to take these tests seriously. This is just one of many problems with this system of "Accountability."

Does your school base graduation on these exams?

Posted by: AGAAIA | May 6, 2010 9:18 AM | Report abuse

The problem seems to be that NCLB encouraged states to give "easier" tests, so that kids score higher, thus making it look like those states are improving, when in fact they were dumbing down the curriculum. If that's the case, then we are in trouble as we have been penalizing states with tough standards and rewarding states with poor standards.

Posted by: celestun100 | May 6, 2010 9:20 AM | Report abuse

Any chance of the Post listing all the states scores on the NAEP?

Posted by: celestun100 | May 6, 2010 9:38 AM | Report abuse

celestun100:

GO TO:

http://nces.ed.gov/nationsreportcard/states/

This data is all available on the NCES website.

Posted by: AGAAIA | May 6, 2010 9:50 AM | Report abuse

@AGAAIA
thanks!

Posted by: celestun100 | May 6, 2010 10:01 AM | Report abuse

Looking at the states that did well, they are all in the southwest. Could it be that their curriculum aligns with the test?

Posted by: celestun100 | May 6, 2010 10:07 AM | Report abuse

As I read the numbers, DC 4th graders saw a 35% increase in proficiency on the NAEP from 2007-2009 and a 22% increase on the DC-CAS. For math it was a 17% increase on NAEP and 40% on DC-CAS. 8th graders saw a 14% increase in reading on NAEP and 26% on DC-CAS. for math it was a 36% increase for NAEP and 32% for DC-CAS. Reading the numbers that way--and I'll admit it is without a Rhee-kicks-puppies-for-fun kind of bias--I don't see evidence of a conspiracy.

Posted by: horacemann | May 6, 2010 11:22 AM | Report abuse

horacemann:

I fail to see what you see...

Between 2007 and 2009 the NAEP data says that
the % change in "At or Above Proficient":

MATH
4th Grade: 3% increase (from 14 to 17%)
8th Grade: 3% increase (from 8 to 11%)

READING
4th Grade: 3% increase (from 14 to 17%)
8th Grade: 2% increase (from 12 to 14%)

Please let me know if you find some other interpretation...

And if you compare the levels of proficiency claimed by the DCPS in their published DC-CAS results for 2009, you will realize how incredible our test system truly is. Absolutely not credible.

GO TO:
http://dcps.dc.gov/DCPS/About+DCPS/Press+Releases+and+Announcements/Press+Releases/Fenty,+Rhee+and+Reinoso+Announce+DCPS+2009+DC+CAS+Scores

MATH (DC-CAS)

Elementary: 49% At or Above Proficient
NAEP 4th Grade: 17% At or Above Proficient
PROFICIENCY GAP: 32% (DC-CAS - NAEP)

Secondary: 40% At or Above Proficient
NAEP 8th Grade: 11% At or Above Proficient
PROFICIENCY GAP: 29% (DC-CAS - NAEP)


READING (DC-CAS)

Elementary: 49% At or Above Proficient
NAEP 4th Grade: 17% At or Above Proficient
PROFICIENCY GAP: 32% (DC-CAS - NAEP)

Secondary: 41% At or Above Proficient
NAEP 8th Grade: 14% At or Above Proficient
PROFICIENCY GAP: 27% (DC-CAS - NAEP)

GO TO:
http://gfbrandenburg.wordpress.com/2010/04/15/another-credibility-gap-looks-like-rhee-reinoso-are-gaming-the-tests-to-try-to-look-good/

Guy Brandenburg has done a great job to illustrate this point.


Posted by: AGAAIA | May 6, 2010 12:21 PM | Report abuse

Just so I don't mix anyone up, when I say the southwestern states did well, I was looking at the color coded map at the site, not the numbers themselves.

Posted by: celestun100 | May 6, 2010 12:57 PM | Report abuse

Another Important Point About the DC-CAS:

The demographic profile of the test population has shifted dramatically under the Chancellor's leadership between 2007 and 2009.

Here are the figures:

% Change in Population by Race
2007 - 2009

4th Grade:
African American: -21%
Hispanic: +3%
White: +8%
Asian: +10%

8th Grade:
African American: -36%
Hispanic: -8%
White: -22%
Asian: -6%

In other words, the poorest predictable scoring populations, namely Blacks and to a lesser degree Hispanics, have decreased in number relative to the predictably higher scoring White and Asian population.

Furthermore, There has been an amazing reduction of the number of 8th Grade Blacks who took the DC-CAS between 2007 and 2009. There were 3,809 students of all races that took the DC-CAS in 2007, compared to just 2,521 in 2009. Of this reduction of 1,288 students taking the test, there were 1,192 fewer 8th Grade Blacks, which means that a combined 104 fewer Hispanics, Whites and Asians took the DC-CAS in 2009. With this dramatic change in testing demographics, it is clear that their would be a significant increase in scores and proficiency levels even if the individuals who took the tests made no progress at all.

I am not qualified to do the statistical regression required, but I am sure someone out there can demonstrate just how this demographic shift would likely adjust scores, within certain probabilities.

Posted by: AGAAIA | May 6, 2010 1:03 PM | Report abuse

A -- The point is not that there's a gap, it's that the percentage increases in proficiency are fairly consistent and do not point to a gaming of the system, as you claim. The DC-CAS may easier than the NAEP, but it is roughly the same amount easier in 2007 as 2009. Happy to be shown wrong, but at least thought it worth looking at it from a different perspective.

Posted by: horacemann | May 6, 2010 1:12 PM | Report abuse

1.) I thought Race to the Top funding was given to the states which came up with the best plan to track student data and promote initiatives and approaches to improve educational outcomes. I don't thing it's just a prize for those states which are already doing a terrific job. These states don't need to race to the top--they can just take a leisurely stroll.

2. For those of you using statistics, please differentiate between percent changes and changes in percentage points. From AGAAIA's post above, you might assume that the number of Asian 4th graders (+10%) taking the DC-CAS jumped from something like 3% to 13% of the total, when it seems that it more likely increased from 3% to 3.3%. It would have been more balanced and less startling to have used change in percentage points year over year. Using percent changes without displaying the underlying data gives too much weight to de minimus subgroups which haven't much effect on the whole.

Posted by: gardyloo | May 6, 2010 2:26 PM | Report abuse

horacemann:

It is not that the tests are easier, it is that any score that would be considered proficient in one rating system, could be so below standard in another. We are still last among a very sorry lot.

And the Proficiency Gap getting larger from 2007 to 2009 should tell you that DC is in fact, Gaming the System. The NAEP test results nationwide have shown a very slow steady rise, and the test development has been much more carefully designed to provide good comparisons between the testing years studied.


gardyloo:

You are correct, but the overall population distribution is changing dramatically towards one that will score better on standardized tests.

If Blacks make up the majority of the DCPS population, and the Black population is decreasing by the largest percentage, by definition they are decreasing in proportion to the rest by huge amounts.

I can get the data for actual populations and proportions, but it will have to wait, as I must pick up my kids from school...

Posted by: AGAAIA | May 6, 2010 4:30 PM | Report abuse

gardyloo,

Race to the Top is about encouraging Arne Duncan's version of reform, not true reform.

Duncan is trying to encourage the same artificial reform in the country that he did in Chicago.

As far as I know none of his reform is based on evidence.

Posted by: jlp19 | May 6, 2010 5:04 PM | Report abuse

Attempts to compare United States students to students in other countries with so-called "world class" standards are specious. No other country dumps special education kids in regular classrooms. In many countries children are tracked in ways that are illegal here. Grading schemes such as this one which compares results of US schooling to other countries is a matter of comparing apples to oranges.

Posted by: buckbuck11 | May 7, 2010 8:37 AM | Report abuse

The comments to this entry are closed.

 
 
RSS Feed
Subscribe to The Post

© 2010 The Washington Post Company