Network News

X My Profile
View More Activity

New Ammo For Charter Debate

The New York City Charter Schools Center is releasing Tuesday a new study showing that students who attend the city's charter schools do significantly better than similar students not picked in the annual random lotteries for charter school places.

This kind of randomized study, often called the gold standard for educational research, is designed to test if students subjected to a new approach--in this case, independent public charter schools--outperform students just like them who do not get the same treatment.

Charter school critics have argued that students in those schools have unusually motivated parents who take advantage of the charter school option, and it is the parents' personal qualities that cause those students' academic success. But if, as this study suggests, children of those motivated parents who actually attend charters do better than children of similarly motivated parents denied charter school places by chance, then charters must have a beneficial effect.

This study, "How New York City's Charter Schools Affect Achievement," is certain to be closely examined and criticized, since charter school research has become a very argumentative academic pursuit. The lead researcher on this study, Stanford economist Caroline Hoxby, is one of the most energetic and prolific researchers in the field. Her results have tended to support the pro-charter school side, so get ready for the other side to start finding flaws in the details of what is a very complex research project affecting 43 New York charters. . .

By Jay Mathews  | September 22, 2009; 4:01 AM ET
Categories:  Jay on the Web  
Save & Share:  Send E-mail   Facebook   Twitter   Digg   Yahoo Buzz   StumbleUpon   Technorati   Google Buzz   Previous: Elite Schools Don't Make Elite People
Next: Testing Vet Reveals How to Fix Standardized Tests


Students themselves are probbaly more motivated to be in a charter school than a public school. I do not know of social and academic pressures in a NY public school, but it seems that there are a great deal less negative pressures in a charter school and more pressure to produce positive outcomes. Too many students in public schools just want to get by.

Posted by: ericpollock | September 21, 2009 9:39 PM | Report abuse

Please read Jeff Henig's book Spin Cycle before you take any single study as the cat's meow and the final word. Research works best by accumulation, not by mythical trump card.

Posted by: ShermanDorn | September 21, 2009 11:59 PM | Report abuse

"This study, "How New York City's Charter Schools Affect Achievement," is certain to be closely examined and criticized,"

Which, I take it, you haven't done.
I guess you must hard at work framing your Shaw story, but needed a little break.

Posted by: edlharris | September 22, 2009 12:36 AM | Report abuse

Did you say the New York Charter Schools Center is releasing this information? Oh, Jay!

Posted by: Linda/RetiredTeacher | September 22, 2009 11:20 AM | Report abuse

Mr. Matthews,

I understand why many people look at these lotteries as the gold standard for studies, but there remain serious problems with them.

First, there is no placebo here. The lottery "losers" are not under the illusion that they are getting the desired treatment. So, you cannot tell how much of the difference is due to dashed hopes or simply belief by the students or their families that one school is simply better than the other.

Second, obviously these kinds of studies never seem to account for peer effects. That is, we already know that stronger fellow students (i.e. high scoring, more cultural capital, higher SES, what have you) lead to higher scores. And we know that charter students are not representative of the the non-charter public school population. The real question that we are looking to answer is whether charterness makes a difference. But if you don't control for peer effects, then you are not actually looking at charterness.

Third, not all charter schools are oversubscribed. We would expect that the better ones, by some measure, would be mostly likely to be oversubscribed. Well, if we think about it a bit more deeply, we'll say that the ones that are MOST likely to be oversubscribed would be the ones with the biggest gap in quality with their nearest non-charter schools. This biases the study to below average non-charter schools and above average charter schools, meaning that neither set of schools included in the study are representative.

Given these kinds of issues -- and from what I've seen, Hoxby's works is often subject to sample bias isssues -- how should we read this report?

Posted by: ceolaf | September 22, 2009 2:09 PM | Report abuse

For edharris: You are right. This post was designed just to alert readers to the existence of this new research, and why it was different from most studies. I would have liked writing more. I always do. But my Web site gurus are trying to train me to accept the fact that every post does not have to be column length, and that in the end readers will be grateful for a good mix of small heads-up pieces and my more typical lengthy rants. And for those who haven't seen the fine Henig book, here is the link to my column on it.

Posted by: Jay Mathews | September 22, 2009 3:30 PM | Report abuse

For ceolaf: I asked Hoxby yr good question about randomized studies leaving out the bad charters that can't get enough applicants to have lotteries. She said that in this case, it is very rare to find a NYC charter that is not oversubscribed and does not have a lottery. I didn't know that myself. She told me after I posted that blog item that the sample her computers crunched includes about 40,000 students who applied for charters and participated in lotteries.

Posted by: Jay Mathews | September 22, 2009 3:36 PM | Report abuse

Thinking about ceolaf's multi-faceted comment as I went to get a drink of water, I realized his point about peer effects deserves a full column. First, I would be grateful for any data he can provide me at showing that "charter students are not representative of the the non-charter public school population." I have not seen such data myself. The data I have seen suggests they are pretty much the same in every measurable way as the other kids in their neighborhoods.
Second, what's wrong with counting peer effects (if there are no significant demographic differences between charter and regular school kids) as part of charterness? This isn't like going to MIT, where everybody is up in the 2100-plus SAT stratosphere. Urban charter kids, at least coming in, are below average. Their distinctive peer effect is, as you say, they are in a school their parents chose, and thus are likely more in tune with what is happening there. Shouldn't schools of choice get some credit for that feeling as something they have created? We have lots of regular urban public school magnets these days that are chosen by parents because of special qualities designed to encourage those choices, and that don't have any special entrance requirements. I wouldn't deny them a chance to claim the benefits of being open to all parents who want their kids to go there.

Posted by: Jay Mathews | September 22, 2009 4:05 PM | Report abuse

I came across another study, and comments related to it that demonstrate charter schools are "mixed bag"
you can click on the study itself within the blog I am posting:

Posted by: researcher2 | September 23, 2009 6:24 AM | Report abuse

The comments to this entry are closed.

RSS Feed
Subscribe to The Post

© 2010 The Washington Post Company