The ed report that all 'reformers' should read
Correction: An earlier version of this incorrectly reported that the study said that standardized test scores cannot be used to show progress in student learning. The study says that scores are of limited value in determining causes of improvements in student performance.
Here are two take-away messages for school reformers around the country that are embedded in an important new report about how the D.C. public school system is faring under the 2007 law that gave the mayor control.
* The standardized test scores that have been trumpeted to show improvement in the schools provide limited information about the causes of improvements or variability in student performance. This would be true, presumably, for any school system that use standardized tests as a measure of achievement.
*The accountability mavens who have pushed standardized-test-based accountability systems on schools, students and teachers forgot something important: to create and implement a system to hold themselves accountable.
The report, titled “A Plan for Evaluating the District of Columbia’s Public Schools: From Impressions to Evidence,” was issued Friday by the National Research Council, the research arm of the National Academies, which include the National Academy of Sciences, the National Academy of Engineering and the Institute of Medicine. It is the first in a series of mandatory reports under the Public Education Reform Amendment Act (PERAA), which allowed then-Mayor Adrian Fenty to take control of the city’s public schools and unilaterally appoint Michelle Rhee as chancellor.
My colleague Bill Turque wrote in a Pos story that reading and math scores on the District’s two main tests, the annual D.C. Comprehensive Assessment System (DC CAS) and the National Assessment of Educational Progress (NAEP), have shown what the report describes as "modest improvement" under Rhee and her predecessor, former superintendent Clifford Janey, although elementary-grade scores fell in 2010.
Rhee made the DC CAS the prime currency of her school reform effort to evaluate schools, students and teachers. But the report says that there are so many problems with the test scores that an overall rise in the numbers cannot be pointed to as proof of student learning.
The report says:
“1. The DC CAS is designed to measure students’ mastery of specific academic skills, but determining whether and how the changes in district policies or strategies have contributed to those skills requires additional empirical evidence: the scores themselves do not provide evidence about what accounts for them.
“2. The available scores are averaged across the entire student population, and do not provide information on the status or progress of specific groups: some may be making sharp gains while others are not.
“3. Because DC is a highly mobile district and the student population changes every year, score fluctuations may be the result of changes in the characteristics of the students taking the test, rather than improvements or declines in students’ knowledge and skills.
"Thus, in order to draw any conclusions about the effect of PERAA on student achievement as measured by DC CAS, further study of patterns for types of schools, individual schools, grade levels, neighborhoods, wards, and population subgroups is needed.”
This is an important finding because standardized tests have become the focal point of reform not only in the District but across the country, and the results are being used to evaluate not only kids but their schools and now teachers. Assessment experts have long said that these tests are not designed to do all of the things that are being done with them in the name of reform, but nobody in control of education policy has listened.
The report says: "Looking at test scores should be only a first step – not an end point – in considering questions about student achievement, or even more broadly, about student learning."
It also notes that reformers are happy to evaluate kids and teachers and schools, but not so much themselves. It says:
“PERAA was ... also spurred by impressions of the effectiveness of reforms in other urban districts facing at least somewhat similar economic, social, and historical challenges. Districts in Cincinnati, Minneapolis, New York City, Boston, and Chicago (among others) have focused on the alignment of content and performance standards with curricula, instruction, and other aspects of the school system. They have used data to guide their decisions, emphasizing such goals as improved professional development for teachers and principals; more frequent formative assessments; and the development of a culture of learning and collaboration among teachers. These approaches are widely used and are supported by some promising evidence, but the research literature is not yet settled enough to provide firm guidance on best practices for district reform or evaluation.
“Some districts have also focused on the governance of schools, and a few (e.g., Chicago, Boston, New York City and Cleveland) have given their mayors control over the public schools. Such reforms are designed to 'jolt' the system by changing dysfunctional institutional relationships and giving leaders new lines of authority and accountability. Evaluation of these governance reforms is critical to knowing what really works and what does not, but few cities have made this a priority, so there are neither clear exemplars nor substantial evidence to guide the District as it implements” the mayoral control law.
New York City, schools for example, have been under mayoral control since 2002, and that control has been extended to 2015. Still, the report indicates that the district is one of the school systems with mayoral control that failed to devise an adequate accountability system to evaluate the effectiveness of reforms undertaken under the administration of Mayor Michael Bloomberg.
Still, reformers proceed as if the reforms they have implemented have worked because test scores have gone up. This report lays bare the falsehoods behind this kind of thinking. We can only hope that somebody in control of education policy actually reads it and then gives it some serious thought.
Follow my blog every day by bookmarking washingtonpost.com/answersheet. And for admissions advice, college news and links to campus papers, please check out our Higher Education page at washingtonpost.com/higher-ed.Bookmark it!
| March 8, 2011; 5:00 AM ET
Categories: D.C. Schools | Tags: dc cas, mayoral control, mayoral control of schools, michael bloomberg, michelle rhee, national academies, report on d.c. schools, standardized tests
Save & Share: Previous: Why schools should try things not "research-based"
Next: Ravitch: Jon Stewart and other education heroes
Posted by: GoldiLux | March 8, 2011 6:39 AM | Report abuse
Posted by: efavorite | March 8, 2011 6:55 AM | Report abuse
Posted by: efavorite | March 8, 2011 7:55 AM | Report abuse
Posted by: MisterRog | March 8, 2011 8:41 AM | Report abuse
Posted by: jdman2 | March 8, 2011 8:48 AM | Report abuse
Posted by: jbeeler | March 8, 2011 8:52 AM | Report abuse
Posted by: frankb1 | March 8, 2011 9:41 AM | Report abuse
Posted by: efavorite | March 8, 2011 9:50 AM | Report abuse
Posted by: frankb1 | March 8, 2011 9:50 AM | Report abuse
Posted by: frankb1 | March 8, 2011 9:50 AM | Report abuse
Posted by: frankb1 | March 8, 2011 9:54 AM | Report abuse
Posted by: efavorite | March 8, 2011 10:18 AM | Report abuse
Posted by: frankb1 | March 8, 2011 10:19 AM | Report abuse
Posted by: DHume1 | March 8, 2011 10:28 AM | Report abuse
Posted by: paulhoss | March 8, 2011 10:32 AM | Report abuse
Posted by: frankb1 | March 8, 2011 10:43 AM | Report abuse
Posted by: DHume1 | March 8, 2011 10:44 AM | Report abuse
Posted by: researcher2 | March 8, 2011 10:53 AM | Report abuse
Posted by: jlp19 | March 8, 2011 11:48 AM | Report abuse
Posted by: frankb1 | March 8, 2011 11:50 AM | Report abuse
Posted by: nfsbrrpkk | March 8, 2011 11:50 AM | Report abuse
Posted by: paulhoss | March 8, 2011 5:49 PM | Report abuse
Posted by: DHume1 | March 8, 2011 6:33 PM | Report abuse
Posted by: paulhoss | March 8, 2011 9:09 PM | Report abuse
Posted by: DHume1 | March 8, 2011 10:27 PM | Report abuse