Network News

X My Profile
View More Activity

Posted at 6:16 PM ET, 01/10/2011

Does KIPP shed too many low-performers?

By Jay Mathews

My colleague Valerie Strauss, creator and proprietor of the fabulous The Answer Sheet blog on this Web site, recently encouraged a spirited debate over attrition rates at KIPP schools. I wrote my last book, "Work Hard. Be Nice" about the birth and growth of KIPP, the charter school network most successful in raising student achievement. (The official name is now just KIPP, not the Knowledge Is Power Program.)

I still follow KIPP closely. I want this blog to be the go-to place for anyone who wants to keep up with important developments in the network of 99 schools in 20 states and the District. Valerie has graciously agreed to allow me to put those recent KIPP posts from the debate here, so you can easily follow the lines of reasoning and can read my views.

It began with a great post (despite its polite digs at me) by Richard D. Kahlenberg, the Century Foundation senior fellow who has provided much original thinking on how to improve the education of disadvantaged children:

In the recent education debate between Valerie Strauss and Jay Mathews, a question arose about the attrition rates at the highly regarded Knowledge Is Power Program (KIPP) schools. The issue is important because if large numbers of weaker students drop out of KIPP’s rigorous program, it would be highly unfair to compare the test score gains won by the top KIPP students against the scores of all regular public school students – who include KIPP dropouts.
In the debate, Strauss mentioned some studies finding that KIPP schools “have had a very high attrition rate.” Mathews responded by saying it is a “myth that KIPP schools have poor retention rates” and cited a 2010 study that found that KIPP school “are doing about as well as regular schools in their neighborhoods” in terms of attrition.
Who’s right? While I respect Jay Mathews’s grasp of educational issues, on this question, the data overwhelmingly support Valerie Strauss’s skepticism.
In a rigorous 2008 study of five KIPP schools in the San Francisco Bay Area, researchers at SRI International found that an astounding 60% of KIPP students left over the course of middle-school. Moreover, the researchers found evidence that the 60% of students who did not persist through the tough KIPP regimen (a longer school day and week, and heavy doses of homework), tended to be the weaker students.
KIPP supporters, like Mathews, respond that a 2010 study of 22 KIPP schools by Mathematica found that the attrition rates were comparable to nearby high poverty public schools that also have lots of kids leave. Poor people tend to move frequently, so high attrition rates are to be expected at KIPP schools, it is argued.
The big difference between KIPP and regular public schools, however, is that whereas struggling students come and go at regular schools, at KIPP, student leave but very few new children enter. Having few new entering students is an enormous advantage not only because low-scoring transfer students are kept out but also because in the later grades, KIPP students are surrounded only by successful peers who are the most committed to the program.

(At this point Kahlenberg displays a graph showing the attrition rate at the KIPP Bay Area schools. You can find it on Valerie's blog.)

In the comments section of the Answer Sheet blog, when readers pointed out that KIPP schools don’t generally fill students back in, Mathews responded, “KIPP schools DO take in new students beyond the 5th grade.”
This is technically accurate, but as the figure above suggests, the vast majority of students enter during the 6th grade (a natural time to enter middle school) and then the total number of KIPP students in 7th and 8th grade falls precipitously.
The KIPP Bay-area schools cannot be dismissed as an outlier on the KIPP attrition question. Columbia University researcher Jeffrey Henig’s 2008 review of several studies found high attrition rates at a number of other KIPP schools.
It may well be, in fact, that high attrition rates are a key explanation for KIPP’s success in raising test scores. When KIPP tried to take over a regular public school – where the students are not self-selected, but are assigned to the school; and where students not only leave, but large number of students enter — KIPP abandoned the field after just two years. KIPP long ago realized that what we charge regular public schools with doing is far more difficult than what KIPP seeks to do.

There were many good comments, both con and pro, attached to Kahlenberg's piece. Here is one that caught my eye. (Ignore the misspelling of my last name. I am sure it was not intentional.

The difference between Valerie Strauss and Jay Matthews is the difference between a real reporter who actually digs for information and uses it objectively and a reporter with a book to sell. Posted by: adcteacher1 | January 4, 2011 8:02 AM

Still groggy from excessive football watching over the weekend, it took me a few days to catch up with the debate. Here is the comment I posted:

I have great respect for Rick and his work, but he is putting far too much weight on data from one KIPP region, and four-year-old data at that. (The Bay Area study came out in 2008, but it was reporting things that happened a couple of years before.) Henig says that some KIPP schools have high turnover rates, but that is not the same thing as saying they are throwing out kids. All the rest of the available evidence shows that KIPP schools on average lose no more kids than other local schools do, and in some cases, once the KIPP school is well established, they lose fewer students.
I think this discussion is fine, with many good points, but it irks me that commenters will suggest, based on one data point, that KIPP is kicking out low performing students. That is wrong. As one commenter pointed out, most of those students leave for the usual reasons, the parents move or they just prefer another school. One of the Bay Area schools lost many kids its first year because they recruited in one neighborhood, and then had to move across the city because they could not keep the facility they had in that neighborhood. Their turnover rate is less now.
As for the depth of reporting in my KIPP conclusions, Valerie is a terrific reporter, and I love her blog. But I think she will agree that on the subject of KIPP, my eight years of research for that book, including visits to more than 40 KIPP schools, has allowed me to learn more about that particular subject than she or Rick has, as good as they are. It is my specialty, and I appreciate the chance to defend what I know in exchanges like this. And don't forget, Rick's review of my book was a rave! Posted by: Jay Mathews | January 7, 2011 12:48 PM

Reading that comment now, I realize I was wrong to suggest that Kahlenberg was accusing KIPP of kicking out kids. (Although other commenters often do say that.) His point was more subtle. He was suggesting that attrition at KIPP, for whatever reason, was making its achievement gains look better than they actually were. I was saved from making more self-congratulatory, deceptive comments by a post from Jonathan Cowan, the KIPP Foundation's chief research and innovation officer, and Steve Mancini, KIPP's national spokesman, who addressed Kahlenberg's concerns:

On Jan. 3, The Answer Sheet featured a guest post by Richard Kahlenberg that highlighted attrition in KIPP schools.
We respect Mr. Kahlenberg’s right to question KIPP’s results, and we welcome healthy debate about the merits of KIPP’s philosophy and model. However, it is also important to clarify the fact base around the issues he raises.
At KIPP, we have a long standing commitment to transparency, continuous learning, and improvement. As such, we are always improving our data collection and reporting processes in order to share our successes and challenges. We focus on understanding the "health" of our schools: Are we serving the students who need us? Are our students staying with us? Are our students making academic progress? Are we fulfilling our promise to get kids to and through college? Are we creating a sustainable model?
Over the last few years, we have begun publicly reporting our performance as it relates to these questions. For instance, we publish our student mobility data in our annual Report Card, to illustrate whether our students are, in fact, staying with us. Our success depends on being held accountable for the results we produce for our kids.
In order to address specific criticisms raised in the piece, we ’d like to clarify Mr. Kahlenberg’s conflation of KIPP’s attrition statistics and our policies on “backfilling” empty student spots. In fact, these are two entirely separate issues, and should be addressed individually:
Assertion 1: KIPP’s success is due to high attrition and the fact that the “weakest” students leave.
Mr. Kahlenberg mentioned the June 2010 report by Mathematica Policy Research, but claimed that it does not tell the whole story when it comes to student mobility. In fact, the Mathematica report is very comprehensive, looking at 22 new and full-fledged schools over four years.
As Mr. Kahlenberg stated, the study found that attrition rates at KIPP schools nationwide were not systematically higher or lower than at comparable schools—some schools had higher attrition, some lower, some the same.
But the Mathematica report also had a second finding that Mr. Kahlenberg did not highlight: The vast majority of KIPP schools had a significant impact on achievement for all students who had ever attended, even if they didn’t complete all four years. Students who left the 22 schools during the study period were still counted in the report, which means the high achievement researchers found was not just a result of attrition. In fact, in conducting the analysis this way, Mathematica is holding KIPP accountable for all the students it ever enrolled, whether they stayed or left.
In his post, Mr. Kahlenberg relied on a study of KIPP Bay Area schools, published by SRI International in 2008, that found those schools to have unusually high levels of attrition. We absolutely agree that this study was rigorous and its findings are valid.
However, it was based on data from just five KIPP schools over a three-year period, and only one of those schools had reached full enrollment at the start of the study period. Thus, the SRI study does not account for how attrition rates at those schools have fallen as these KIPP schools have matured over the past four years.
Assertion 2: KIPP middle schools have high test scores because they do not enroll students after sixth grade.
Mr. Kahlenberg claimed that the reason Mathematica’s attrition results are flawed is because KIPP schools do not accept new students to make up for the ones they lose. He acknowledged that KIPP does take in new sixth-graders, but claimed that this is because sixth grade is “a natural time to start middle school.”
However, it is not the case that KIPP cuts off enrollment after sixth grade. Many KIPP middle schools, including those at KIPP DC, now regularly enroll new students at all grade levels, fifth through eighth. KIPP’s high schools also take students at all levels, from ninth to twelfth grade.
As more schools are reaching full enrollment and sustainability, this issue of “backfilling” classes is also subsiding.
The SRI study data Mr. Kahlenberg cites cuts off in 2006-07, when the eighth grade class was at 55% of the starting size of the entering fifth grade class. But that data is now several years old, and those numbers have improved dramatically. As of 2010-11, the KIPP Bay Area eighth grade class is at a full 86% of its starting fifth-grade size. We are working hard to increase that percentage even farther, at KIPP Bay Area and in all other regions.
As KIPP continues to grow, moving from start-up to sustainability, we have seen significantly reduced attrition rates and had success enrolling students at all grade levels. We remain focused on continuing to improve in these areas so we can set ever more students on the path to college and a better future.

Kahlenberg's response said in part:

Cowan and Mancini don’t dispute the data, but say that KIPP has gotten better at reducing attrition since the 2008 study was published, and that “many KIPP middle schools, including those at KIPP DC, now regularly enroll new students at all grade levels.” This is a very welcome development but could use some elaboration.

Cowan and Mancini say that KIPP is committed to transparency, and in that spirit, I’ll end with three questions:
1. How many students are now entering KIPP schools across the country during the seventh and eighth grades? Today, what is the aggregate difference between the size of the sixth grade KIPP classes two years ago and the eighth grade classes today? And how has the new influx of students in seventh and eighth grade affected KIPP test scores?
2. Which groups of students (by race, gender and income) are most likely to leave KIPP?
3. If KIPP wants to put the self-selection, attrition, and intake issues to rest, why doesn’t it simply start taking over regular public schools, educating the students who happen to live nearby, including those who move in during the course of middle school?

These are good questions, which I suspect the KIPP people will answer eventually. As a reporter for the last 45 years, I have a much experience trying to drag useful information out of large organizations, particularly about schools. KIPP ranks with the very best in responsiveness. If anyone has good questions about KIPP, or anything else I write about, that have not been adequately answered, just send them to me at mathewsj@washpost.com, and I will see what I can do to get the information.

Kahlenberg's last question is the most intriguing. KIPP has never tried hard to take over an existing school. The brief takeover he mentions, which occurred in Denver several years ago, was half-hearted and never had strong support among national KIPP leaders. It was a stop-gap situation with few good alternatives, and it was difficult to get the local cooperation necessary, KIPP people have told me. They said they never found a school leader who met their standards and was willing to make a long-term commitment to run the school.

When I raise the general question of taking over regular schools, KIPP people usually say it is hard enough to make their charter schools successful. I think they would agree with Kahlenberg that turning around a regular urban school is harder than creating a successful urban charter school, but would add that creating a successful urban charter school is still extremely difficult. The data prove them right. Nobody has done as well as KIPP in raising student achievement, and on average urban charter schools nationally are no better than the regular schools in their neighborhoods.

KIPP people also suggest that they would not attempt to fix a regular school unless they were allowed to use the tools they feel have been vital in making their charters successful. One KIPP leader once told me that KIPP could turn around a local school only if it was allowed to hire and fire faculty without regard to existing rules, something that was not going to happen.

KIPP would also insist, I suspect, on being allowed to institute its nine-hour school days, required summer school, homework requirements and discipline measures, and be free to change them when necessary. In other words, KIPP wouldn't try to take over a regular school unless allowed to run it as it runs its own schools -- with the only difference being that all neighborhood students would be able to attend. KIPP now takes all students who sign up for their schools and win the lottery if there is not enough space.

I wager some district superintendent, maybe a former KIPP teacher, will some day persuade a KIPP school leader to take a regular school under the conditions above, and see what happens. Rick Kahlenberg makes the good point that such an experiment would help KIPP, and the rest of us, learn what might really help our urban schools to change for the better.

Read Jay's blog every day, and follow all of The Post's Education coverage on Twitter, Facebook and our Education Web page.

By Jay Mathews  | January 10, 2011; 6:16 PM ET
Categories:  Jay on the Web  | Tags:  Jonathan Cowan, KIPP, KIPP attrition rates, KIPP taking over a regular school, Richard Kahlenberg, Steve Mancini  
Save & Share:  Send E-mail   Facebook   Twitter   Digg   Yahoo Buzz   Del.icio.us   StumbleUpon   Technorati   Google Buzz   Previous: Learning inspired by teacher and textbook errors
Next: Obama on No Child Left Behind: Get me rewrite

Comments

Jay,

I think both you and Valarie had extraordinary points about KIPP schools. My sister has worked in a KIPP school, here, in New York for the past eight years, and beyond her school's high test scores, I'm incredibly impressed with some of other co-curricular courses, such as African drumming, a few foreign languages, and voice lessons. These types of courses, in my opinion, outline a robust liberal arts curriculum. I think you guys need to touch on a-little more than test score data and delve into the intricate things that make each KIPP school unique.

On another note, it doesn't hurt, as journalists and writers, to do a spell check. After all, it's free, and lately, most of your posts have been riddled with grammatical errors.

Posted by: rasheeedj | January 11, 2011 7:13 PM | Report abuse

sigh. editors have often said the same thing to me.

Posted by: jaymathews | January 11, 2011 9:10 PM | Report abuse

This is about commitment and proper behaviour. KIPP is producing results for those wanting it and the ability to get it. The liberals and public school apologists like to shoot KIPP down instead of looking at their own house as it burns.

Jimmy Kilpatrick

Posted by: JimmyKilpatrick | January 12, 2011 6:21 AM | Report abuse

Like many programs set aside for those "wanting" in, the bottom line depends on, as Jimmy posted, "behavior." Education demands discipline. That is both a learned "behavior" and added with an enforced/required behavior.

Those dropping from a more "visible" or more "watched" program like KIPP are probably less disciplined and parental oversight is not as strong (without survey data, this "appears" to be the rationale.) Those students excelling and retained are possibly children of more engaged parents.

Public schools however, are (I dislike the term but I'll use it here) mandated melting pot of learning. Discipline is more voluntary, more widespread, and more diverse. It cannot be mandated like attendance because the administrations "KNOW" discipline is a variable and parental engagement is also a variable.

The unintended consequences of KIPP is Darwinism by holding parents and children (dare I say customers) accountable and responsible for the child's discipline and behavior.

Posted by: jbeeler | January 12, 2011 8:04 AM | Report abuse

Like many programs set aside for those "wanting" in, the bottom line depends on, as Jimmy posted, "behavior." Education demands discipline. That is both a learned "behavior" and added with an enforced/required behavior.

Those dropping from a more "visible" or more "watched" program like KIPP are probably less disciplined and parental oversight is not as strong (without survey data, this "appears" to be the rationale.) Those students excelling and retained are possibly children of more engaged parents.

Public schools however, are (I dislike the term but I'll use it here) mandated melting pot of learning. Discipline is more voluntary, more widespread, and more diverse. It cannot be mandated like attendance because the administrations "KNOW" discipline is a variable and parental engagement is also a variable.

The unintended consequences of KIPP is Darwinism by holding parents and children (dare I say customers) accountable and responsible for the child's discipline and behavior.

Posted by: jbeeler | January 12, 2011 8:06 AM | Report abuse

What I take away form this discussion is that one of the factors making KIPP schools successful is the fact that students are there by choice.

But what is keeping districted public schools from instituting choice on a smaller scale? Many middle schools divide the grades into "teams," but the teams offer the exact same teaching and curriculum. Why couldn't they offer different things and have students and parents _choose_ the best fit for them? Why do they insist on a one-size-fits-all education?

Many schools in NYC are divided into themed academies. Why couldn't more schools do the same?

Instead of asking "Why can't KIPP take over an existing public school," why not ask, "Why can't existing public schools be more like KIPP?" After all, organizing a school differently is a lot less costly than putting smartboards in every classroom.

What is preventing districted public schools from offering a diversification of options?

Posted by: hainish | January 12, 2011 9:06 AM | Report abuse

@hainish- Where I teach, Philadelphia, there has been an increase in "choices" for high schools. These programs are almost exclusively small and directed towards a particular focus, such as culinary arts. However, most open admissions high schools are also *selective* admissions as well. As a result, most students in Philadelphia who apply for admissions end up in regular, local high schools. The figure is above 70%.

In short, this is *not* a solution to big-city education problems although it provides a benefit for some students and their families. When you have 160,000+ students, you have to really look at the whole system. Sometimes, I think it would be better to create disciplinary wings in clusters of schools so that the rest of the students and teachers could just focus on learning. There's a lot you can accomplish when you aren't operating in an environment of disorder and violence. I bet many principals, teachers, parents and students could develop some exciting programs on their own if behavioral issues didn't drain so many resources.

Posted by: Nikki1231 | January 12, 2011 9:25 AM | Report abuse

Jay,

I am starting to think your middle name is KIPP.

Posted by: ericpollock | January 12, 2011 10:23 AM | Report abuse

for hainish---I like the point you make very much.

for ericpollock---Not quite, but I get yr point. Sportswriters produce a lot of copy about LeBron James. Political writers focus on President Obama and Sarah Palin. Health writers do much about the new federal law. It is uncommon, I realize, for an education writer to follow that same pattern, because there is so little public agreement over what is the most interesting story. But rarely has an education reporter been given as much time as I have been given by the Post, over 28 years, to get deep into the education weeds and see what measures up and what doesn't. After nearly a decade looking at KIPP, I am convinced it is the most interesting education story there is, with consequences for many other issues, so you are going to hear much about it from me. The Internet, with its infinite room for copy, lets me do that, and I am going to keep up with it, and the other topics that are of particular interest to this blog, like AP, IB, raising the level of low-income schools, and the work of particularly effective teachers and schools.

Posted by: Jay Mathews | January 12, 2011 11:34 AM | Report abuse

Good post that synthesizes a very good and very important debate.

Jay -- I understand your enthusiasm for KIPP and, as I've said before, I respect the hard work they do. Clearly they provide a valuable service for those who come through their doors.

The unanswered question, alluded to by others, remains whether or not KIPP really is a scalable, viable solution for urban education or if it is destined to remain a niche charter network that pecks at the margins, helping only a relatively small number of hard-working urban kids. As you said in the last paragraph, we'll need someone bold or crazy enough to put KIPP into a neighborhood school to see what happens.

Posted by: joshofstl1 | January 12, 2011 12:04 PM | Report abuse

Nikki, I'd hardly take a city offering small, specialized programs along the lines of culinary arts as an example of what I'm talking about.

Disciplinary areas (or holding areas, or classrooms that you go to when you get kicked out of the program you chose) would be a very good strategy. Something happens when you give a people a choice. They become a lot more invested in what they have chosen.

Posted by: hainish | January 12, 2011 12:20 PM | Report abuse

for joshofstl1---You have it exactly right. That is why I am so eager to keep following it, and other efforts like it, and see what happens. KIPP and enterprises like it have many limits and flaws, but they are the most effective organizations we have at the moment in raising achievement for disadvantaged kids. Hopefully they will improve, and inspire others to do better.

Posted by: Jay Mathews | January 12, 2011 1:53 PM | Report abuse

@hainish- Philadelphia and NYC are already doing just what you propose. What you fail to see is that *scale* in big-city school districts is a huge problem. This is also the problem with so many charter schools like KIPP. It all looks great when you are talking about a few schools with small enrollments, which is precisely how KIPP operates in Philly as well as D.C. and other cities.

What happens when you apply dramatic changes across large-scale populations that don't have built-in selection mechanisms? What results do you get when you can't offer students and families "the local school down the street" if they don't adhere to policies or can't hack it academically?

I can give you one example in Philadelphia. I stated that *most* of the open admissions high schools were selective. One veers from this: High School of the Future. Founded with money from MicroSoft, you would think this school would be a success. It is not. One of the biggest problems is the fit between its enriched technology-based curriculum and its students population, a majority of which is drawn from the poor, West Philly community where the school resides. They are simply not prepared academically.

There is one notable CMO that takes over schools in Philadelphia and has a good success rate: Mastery Charter. They now have 6 or 7 schools and have just this year taken over a large, poor, violent and failing elementary, Smedley Elementary. From what I've heard, they are doing quite well and quelling the disorder that persisted there for years. It will be interesting to see what academic gains were made this year. It will also be interesting to note how many students left the school this year, how many were ELL and special ed (a significant population at the school). This information is tracked in PennData and, like KIPP, Mastery has a history of problems retaining special ed students.

In a recent article, many parents interviewed were thrilled with the rigor at Mastery, others were already considering pulling their children out of Smedley, despite the calm school climate. They didn't care for the academic pressure put on their children and the "bottom line" approach. I guess they didn't like to hear that their children were years below grade level and had a lot of work to do to catch up.


Posted by: Nikki1231 | January 13, 2011 5:50 AM | Report abuse

"But what is keeping districted public schools from instituting choice on a smaller scale? "

hahahahaha.

But I see that Nikki has been more polite.

"What results do you get when you can't offer students and families "the local school down the street" if they don't adhere to policies or can't hack it academically?"

This is the fundamental question. Public schools have to educate the kids that don't want to go to KIPP. They can't offer so much choice that there's no school for the kids to go to.

So no, public schools can't be "more like KIPP" (which, despite Jay's enthusiasm, ain't all that). KIPP is able to exist as a fringe of public schools. That's its function.

Posted by: Cal_Lanier | January 13, 2011 1:26 PM | Report abuse

Post a Comment

We encourage users to analyze, comment on and even challenge washingtonpost.com's articles, blogs, reviews and multimedia features.

User reviews and comments that include profanity or personal attacks or other inappropriate comments or material will be removed from the site. Additionally, entries that are unsigned or contain "signatures" by someone other than the actual author will be removed. Finally, we will take steps to block users who violate any of our posting standards, terms of use or privacy policies or any other policies governing this site. Please review the full rules governing commentaries and discussions.




characters remaining

 
 
RSS Feed
Subscribe to The Post

© 2011 The Washington Post Company