Network News

X My Profile
View More Activity


Posted at 12:02 PM ET, 01/ 7/2011

What really helped Florida's test scores

By Valerie Strauss

I've recently published a number of posts about school reform in Florida, which has been driven over the past decade by Jeb Bush, who was governor there from 1999 to 2007 and who still has a great deal of influence over education policy in the Sunshine State, and, increasingly beyond. Florida's reforms revolved around an expansion of standardized testing and choice (vouchers, charter schools, etc.), and it is these measures that are often given credit for a rise in scores on the National Assessment of Educational Progress.

In this post, research analyst Michael Martin looks at what reforms really impacted the test scores. Martin works for the Arizona School Boards Association; this post is his own work and not approved or endorsed by the association.

By Michael Martin
Many efforts to promote ideas taken from the school reforms in Florida often are a classic example of “truthiness,” defined as believing what one wants to be true instead of what is actually true.

Florida did achieve considerable improvements in student achievement during the last decade. But it is important to see when the gains occurred and where they occurred.

Florida had an overall Grade 4 reading score of 205.7 in 1998, which was almost the same as the 204.9 score in 1994 on the National Assessment of Education Progress (NAEP) tests. The score in 2002 was 214.4, rising to 219.5 in 2005, and 223.5 in 2007, and 225.7 in 2009. Thus from a less than one-point change in 1998, Florida saw an almost 10-point change in 2002, a five-point change in 2005, a three-point change in 2007 and a two-point change in 2009.

Clearly something significant occurred to improve reading test scores in Florida between 1998 and 2002 that continued at a much lower level in subsequent years.

The No Child Left Behind Act could only have had a role in diminishing the gains, because its implementation occurred after 2002. Similarly, the voter-passed class size reduction initiative in Florida occurred in November of 2002. Mandatory Grade 3 retention was a law passed by the Florida legislature in 2002 and began in the 2002-03 school year. These reforms can only be associated with the declines in test score gains.

Averages can sometimes be deceptive. Looking at the NAEP scores marking the 10th percentile low end versus the 90th percentile high end provides an insight into what occurred.

Grade 4 reading scores at the 90th percentile went up 4 points from 1998 to 2002, but at the 10th percentile scores went up nearly 15 points. Thus the 10 point overall average gain in Grade 4 reading scores on the NAEP from 1998 to 2002 was primarily a gain at the lowest scoring levels, and Hispanic scores at the 10th percentile went up 19 points and African-American scores went up 17 points.

Something did happen in Florida in 1999 that specifically targeted low scoring students. In 1999 Florida created its “Assistance Plus” program targeting additional resources at schools rated as “D” or “F” in student test scores.

In an October 2002 status report on the Assistance Plus program, it noted that 64 schools were targeted for managerial assistance, including training teachers for “Effective Reading Practices” and funding reading coaches as well as coordinating activities with local college Assistance Plus Teams. The status report noted that “over $25 million statewide” in fiscal assistance was provided to these 64 schools, or about $390,000 per school.

The state created regional school improvement facilitators who were assigned failing schools to work with. These failing schools were paired with higher performing schools to provide peer models and assistance. In addition, colleges provided research-based assistance for implementing school improvement programs.

The schools also began working with other community organizations and administered community outreach programs, including climate surveys in languages of the parents. The Achievement Plus program perfectly fits the time and target of when and where Florida’s test scores improved.

From 2002 to 2005 the gains at the lowest scoring levels in Florida continued. At the 90th percentile NAEP Grade 4 reading scores increased by only 3 points, but at the 10th percentile scores increased over 10 points, with Hispanic scores at the 10th percentile increasing almost 15 points and African-American scores increasing over 12 points.

It is likely that the de-emphasis of the Assistance Plus program in later years accounts for the diminishing increases in Florida reading scores in more recent NAEP tests.

After 2005, the 10th percentile Grade 4 reading scores continued to show improvement, but at about half the levels of previous years. In fact, 90th percentile scores in 2009 had actually fallen by about half a point, but 10th percentile scores increased over 5 points while the overall average increased only 2 points.

People who claim various programs were responsible for the improvement in NAEP test scores in Florida over the past decade must explain why their improved NAEP reading scores primarily occurred among the lowest scoring students while other student scores largely stagnated, and why those increases were most dramatic from 1998 to 2002, diminishing afterward.

Targeting the lowest scoring schools in Florida for funding increases through Assistance Plus also meant targeting these funds at low-scoring minority students, and low-scoring minority students had the largest test score improvements. It is the large increases in 10th percentile test scores that created Florida’s much touted reduction in the achievement gap between minorities and majority students.

Targeting additional resources toward increased management training and professional development among teachers dealing with the most difficult students to teach makes simple common sense. Helping schools and teachers dealing with significant problems in Florida by providing additional human and fiscal resources fulfills Occam’s Razor for the best explanation of Florida’s success over the past decade.

-0-

Follow my blog every day by bookmarking washingtonpost.com/answersheet. And for admissions advice, college news and links to campus papers, please check out our Higher Education page at washingtonpost.com/higher-ed Bookmark it!

By Valerie Strauss  | January 7, 2011; 12:02 PM ET
Categories:  Guest Bloggers, School turnarounds/reform, Standardized Tests  | Tags:  florida reform, jeb bush, jeb bush and school reform, naep scores, national assessment of educational reform, school reform  
Save & Share:  Send E-mail   Facebook   Twitter   Digg   Yahoo Buzz   Del.icio.us   StumbleUpon   Technorati   Google Buzz   Previous: What 2010 education research really shows about reform
Next: Why Fairfax's textbook recall is surprising

Comments

Because the programs get more focus and the kids are IN the program, that transfer of oversight helps the children. Children behave differently when watched. Most behave better.

Posted by: jbeeler | January 7, 2011 12:52 PM | Report abuse

This current focus on data is mind-numbing for teachers and administrators forced to spend countless hours immersed in "data driven" teaching, forced to attend "data mining" meetings... etc.. Imagine how the students feel!!! Data data data... Ironically, reading Martin rattle off data is what public school teachers and administrators often feel as they must spend countless hours immersed in data. Instead of preparing lessons for students or simply tending to the students, they oversee tomes and tomes of "student data" based on all the testing... hmmm ... something is ABSURD in this scenario.

Michael Martin importantly and ironically seems to turn "data" onto itself which to me reveals just how inane it really is. His key points mention that gains in Florida came before 2002 and that thereafter has been downhill. Isn't this totally ironic in that NCLB's impact is post 2002! Martin points this out ... "The No Child Left Behind Act could only have had a role in diminishing the gains, because its implementation occurred after 2002". To add insult to injury... any students who were not in the lowest scoring category seem "unattended to" during the reign of NCLB as their scores stagnated!

I just wonder when this madness will end and when teachers will actually be able to "TEACH" in the classroom. This current focus on data would be laughable at best if it weren't for the tragic fact that it is destroying the academic lives of our nation's neediest children!

Posted by: teachermd | January 7, 2011 6:15 PM | Report abuse

As was once said, there are lies, big lies and then statistics.

While the analysis might look correct for the untrained eye, the timeline discredits the narrative. If one goes to NAEP's website and looks for information on grades 8th and 12th, one will read that all the gains from 4th graders had disappear as time passed by. Compare that to the scores from other states that are at the same level as Florida's 4th graders.

If the "Assistance Plus" program was responsible for what this analyst calls "success," then it also is responsible for its ultimate failure. If a program is not able to sustain gains in the long run, then it should not be deemed as successful; in that resides the fallacy of the conclusion by this analyst.

If Jeb Bush wants to claim success for his education policies, now being extended by his surrogate Scott, then other states should evaluate the long term results of his initiatives. The truth is that at the higher levels Florida's education program is a disaster. The best gauge of such failure is the increased number of unemployed youngsters and college students taking remedial courses.

Education is a lifetime effort that doesn't end in 4th grade.

Posted by: redisni | January 7, 2011 8:32 PM | Report abuse

"Clearly something significant occurred to improve reading test scores in Florida between 1998 and 2002 that continued at a much lower level in subsequent years."

Clearly, the person that wrote this is ignorant of statistics, perhaps even innumerate, and the ignorant Valerie Strauss has cut and paste it into her useless blog. Neither of these two people should be allowed to put anything out without being reviewed by a statistician.

1. The NAEP scores are not an absolute scale, they are scaled to a predetermined mean and standard deviation. Therefore, you can't look at the raw numbers and say that an 5 point difference from 204 to 209 is the same amount of improvement as a change from 209 to 214. See how the scoring for reporting works: http://nces.ed.gov/nationsreportcard/tdw/analysis/trans.asp

2. This is so glaring I can't believe these people didn't see it. Even given the complete ignorance of my first point, the 10 pt change from 1998 to 2002 was over 4 years (an average of 5 pts every 2 years.) The author then compares this with a supposedly slower 5pt gain from 2002 to 2005 (an average of 5pts every 3 years). Are you really that mind-bogglingly stupid, or do you just think we are? A different amount of time elapsed!

Go back to school.

3.

Posted by: staticvars | January 9, 2011 10:11 PM | Report abuse

Post a Comment

We encourage users to analyze, comment on and even challenge washingtonpost.com's articles, blogs, reviews and multimedia features.

User reviews and comments that include profanity or personal attacks or other inappropriate comments or material will be removed from the site. Additionally, entries that are unsigned or contain "signatures" by someone other than the actual author will be removed. Finally, we will take steps to block users who violate any of our posting standards, terms of use or privacy policies or any other policies governing this site. Please review the full rules governing commentaries and discussions.




characters remaining

 
 
RSS Feed
Subscribe to The Post

© 2011 The Washington Post Company