Print Columns   |   Web Chats   |   Blog Archives   |  

The Challenge Index--Jay's Response

Here's Jay Mathew's response to my blog post from earlier this morning.

It is kind of slippery here, with all that buttering up. But I refuse to be roasted without a few yelps of protest. The first complaint would come not from me but from the vast majority of Americans who have ever attended public high schools. You say that "using APs as a ranking tool puts enormous pressure on schools to add more and more AP courses, squeezing out creative classes, limiting what the best teachers can do in their classes, and turning high school into a dull grind for too many of the better students."
I ask your readers: When exactly was this Golden Era when high school was NOT "a dull grind for too many of the better students," or for just about everybody, for that matter. Your distaste for the Challenge Index rests on the false assumption that AP and IB courses somehow bring dull rote learning and tedious academic chores to what has been a vibrant atmosphere in our high schools. The opposite is true. Ask AP and IB students why they choose those courses over their schools' other fare, and they will tell you because those courses are better taught, more challenging, more interesting and much more likely to exercise their analytic and critical thinking skills because---unlike most high school course finals---the AP and IB tests are full of thoughtful questions that require written essays that must be graded by human beings.
The AP and IB tests are authentic and incorruptible, to boot. The people who write and grade the exams are experts neither the students nor the teachers know, getting together in grading sessions in which the exam takers are not identified. The standard high school course dynamic is the students trying to find some way to persuade the teacher to reduce the homework and dumb down the test, and the teacher sometimes doing that for fear that too many bad grades will mean angry visits from parents, or the principal. That can't happen in an AP or IB course, since the test is beyond their control. That lets the dynamic become a much healthier team effort, students and teacher working together to get ready for that exam.
You say adding more AP "means eliminating the really cool Shakespeare elective or the uniquely rigorous European History seminar that made that particular school someplace special." Here is my challenge to you, and if you succeed I will do an entire column about what you have discovered. Find me one instance, that I can confirm by calling the school, in which an AP course ever replaced a really cool Shakespeare elective or a rigorous European history seminar. This is one of those urban myths that falls apart if you spend as much time talking to high school teachers.
The only complaints of this sort that I am getting is from a few very expensive and exclusive private schools, like the ones your kids attend and mine attended, where some faculty members fidget under what they consider the iron grip of the AP curriculum. Exactly 13 of them have dropped AP---notice, AP didn't replace the cool Shakespeare seminar, it was the other way around. One public school, Scarsdale High, is thinking of doing the same, but even in their case no AP course has replaced some cool alternative. I think it is fine for such schools to experiment with non-AP alternatives. They can afford it. Their parents are all college educated and the kids will go to good universities no matter what.
But in the real world, high schools lack the resources to build such courses and sustain them on their own. They might get a great teacher who invents a great course that lasts for awhile, but without any backing, such as an exam that no one can dumb down, those efforts usually fall apart. They need AP or IB. The point of the Challenge Index is even when they have these courses, most schools let only their top students into them. To the others, the poor average students, they say, "We're sorry. We know you are going to go to college, but we are not going to let you take a course and test that will prepare you for college." It's idiotic, but it is the majority view. Only five percent of U.S. schools reach the modest Challenge Index benchmark of one AP or IB test per graduating senior.
I am flattered that you think the Challenge Index has caused the growth in AP and IB in the Washington area. What it has done is a bit more subtle. The successful AP and IB teachers and pro-AP and pro-IB school board members I talk to say the list has given them some ammo to use against the many people who buy the majority argument and say, "You're letting those average kids take AP and IB? What a terrible idea!" The good guys in the argument reply, "Yes, and if we give them enough time and encouragement, they can do it." But they can also say, if the political winds blow against them, "Well, if you keep restricting access to AP and IB, we will look bad in the Post, and our real estate values will decline."
Anything that gives more kids a chance to experience the thinking and analysis of a well-taught AP or IB course is okay with me. And I think the more powerful spur for AP and IB around here is that the selective colleges have decided that high school students need a taste of college learning before they get to college, and if they don't have the courage to try it in high school, they are not likely to get into those schools.
It makes for more pressure, sure, but I have yet to find a Washington area high school student who told me that he or she decided to take a few more AP or IB courses so his or her school would look good in the Post. If you find students who say that is their motivation, send them to me and I will write a column about them too. They take AP and IB because those are the best courses in their schools, and they are much better than the equivalent courses I remember back at Hillsdale High in San Mateo, Calif, a hundred years ago.

By Marc Fisher |  December 8, 2006; 12:16 PM ET
Previous: Challenging the Challenge Index | Next: Listener: The Deflating of Air America


Please email us to report offensive comments.

Fisher - you just got owned

Posted by: dc | December 8, 2006 12:42 PM

Jay, you have succumbed to the Fisher disease, the vibrant virus. This manifests itself in the inability to write a column or blog entry without at least one use of the word vibrant. It's too late for Marc, but you may still be saved. Next time you feel the need to use the word vibrant, remember these synonyms- active, animated, colorful, dynamic, electrifying, energetic, lively, peppy, responsive, sensitive, sound, sparkling, spirited, vigorous, virile, vital, vivacious, vivid- and use one of them. Your readers will thank you.

Posted by: crc | December 8, 2006 1:00 PM

As a teacher I have to say that essays are the best way to evaluate students, since AP and IB clsses require students to write essays they are an unconditional good.

The problem I have with the Challenge Index is the problem I have with most ways we evaluate our schools and our students--we use correlative factors.

Correlative factors are useful as a dipstick only to the extent that those who are being evaluated do not know what correlative factors are being used. For example, the Woodcock Johnson test of Achievement (III) uses a list of 20 or so words to measure spelling, a different list for vocabulary, a set of 18 problems for math calculation and other rudimentary tests for other cognitive skills. It would be possible to teach most students the information on the test in an afternoon. The test only has validity because the subjects are ignorant of the specifics they will be asked. If I were to use these lists and sets of problems as a way of ensuring my students ranked as super-geniuses on the WJ-3 then I would be in error. Teaching someone twenty words will not make a first grade speller into a 12th grade speller overnight.

Similarly, AP and IB participation is useful because it correlates to school performance, so long as administrators are not giving special consideration to AP and IB participation as a factor in their decision making. This is no longer true--AP and IB participation is being treated as a causative factor. In other words as an administrator I would take positive action to increase participation knowing I can game out the challenge index results.

In short the index is becoming less useful as the results of the index are used more widely.
Therefore as Mr Matthews index gains currency and poularity, it diminishes in value as a way to rank schools.

Posted by: Chris | December 8, 2006 1:06 PM

The real problem with the Challenge Index is that it is no way a measure of the quality of a school, because it does not measure how students actually perform on the exams. If every child in a school takes 10 AP exams -- and every child fails every one of them -- then the school would likely appear as #1 in the country on the Index, and quite clearly be a terrible school -- perhaps one of the worst.

One may argue that the Index does not necessarily measure school quality, but there is no question that that is the inference commonly drawn from these Post articles. Instead, the Index should reflect the per capita rate of children passing the AP/IB exams. The Post is measuring the wrong thing, completely confusing its readers about actual school quality, and so doing us a real disservice.

Posted by: Challenge Misnomer | December 8, 2006 1:09 PM

The biggest problem with the Challenge Index is how its used. When it's used for articles about the "Best" high schools, it's wrong. As was pointed out in a number of articles, a number of schools that Jay's index calls the "Best" are actually failing under more rigid criteria - NCLB's Annual Yearly Progress; gaps in achievement among different socio-economic groups; droput and test failure rates; etc.

I have a Master's Degree in Statistics from Purdue University (one of the top ten departments in the US, by the way), and because of that background, analyses like the Challenge Index are always fun - you get to try to figure out the originator's bias, and motivation. In this case, reading Jay's response to some of his critics confirms what the numbers reveal - he wants to show that Garfield High in Los Angeles is a "good school". No standard measure shows that, so he just makes up another measure that shows what he wants.

Print it in the Washington Post, go national with it in the Post-owned Newsweek magazine, and voila - all of the social climbers and headline hogs are now demanding more AP classes, and gloating over rankings.

There's nothing wrong with more AP classes, but when you allow schools that are failing by most measures to escape well-deserved criticism because of this "challenge index", then your over-simplification has done a dis-service.

Posted by: Army Brat | December 8, 2006 1:44 PM

I'll repeat a question that I posted with Marc's earlier column. If the AP and IB curricula and methods are so great--and I'll concede that there are studies indicating that they are--why use them to rank schools so that some are "good" and some are "bad" instead of pushing to have states and school districts adopt them as standard for all of their students?

Posted by: prof | December 8, 2006 2:27 PM

Many of the people that dislike Jay's challenge index are confusing winning with education. They want to know which school has the best studends, or which schools are winning the school reputation contest. To them, the best high schools are the ones that have the most winners as students. That's not necessarily a bad thing, the United States' emphasis on sports and networking probably accounts for how our students have come in near last in all the international tests for the last 60 years, but then grow up to outperform their peers in other countries. However, Jay is not interested in measuring this, he is interested in measuring something else. He knows how high schools work, and he wanted to devise a contest to push high schools into giving their average students a better chance to do well in college. His solution is simple, effective, and an amazing example of what a skilled reporter can do to really improve our society.

Posted by: David | December 8, 2006 3:28 PM

I thought that my posting was clear--my objections to the Challenge Index has nothing to do with winners or losers. The problem is that educational decisions are being made--particularly in this area--with the express purpose to improve participation in IB and AP and therby game the Challenge Index rankings. The whole move towards universal Algebra in 8th grade for MCPS is to force a larger number of HS seniors and Juniors into AP math because there will be no non-AP classes that they have not taken.

It msy be that this ends up being good for a greater number of students, or it may result in an erosion in the quality of Algebra and a greater number of drop-outs. What is important though is the discussion is not in terms of what is best for students, it is how do we get more students in AP math by Junior year.

The problem with the Challenge Index is it's influence and the resulting tail wagging dog behavior by administrators.

Posted by: Chris | December 8, 2006 4:22 PM

I wish to speak in defense of the AP system, especially at high schools that are not of the sort that regularly make it to the top of "Best Schools" list. I come from a family that puts a high value on education. When I was younger, I was in a very creative and stimulating magnet/"gifted and talented" program that emphasized both creative and critical thinking, without letting us off easy in terms of the basic reading and math skills that we needed to learn, in a public school in a medium-sized Midwestern city. Before I entered the ninth grade, my family moved to a smaller city in upstate New York that had one public high school. This school had a large student body from a wide range of backgrounds. (Although it wasn't racially diverse, neither was the town itself- but the school served kids from the middle of the city and from way out in the boondocks, and from both wealthy and not-so-wealthy neighborhoods.) In general, I think it was a fairly "average" public high school- which meant it was a shock for me to find out that many students took a very passive approach, doing the minimum of work necessary to get by, and so did quite a few of the teachers. My parents had the means to send me to a private school, even though they weren't super-wealthy, and came very close to doing so because I was so bored and demoralized. But in my junior year I took my first AP course, along with a math class that wasn't AP but was a real step up in challenge compared to its predecessors. I was more challenged and engaged in school again. I took more AP courses in my senior year. The teachers who taught the AP courses did _not_ lead rote and bland classes. There was more discussion and give-and-take in my AP classes than in my other classes. I read and wrote quite a bit more for my AP history and English classes than I had for previous non-AP classes. In my math and science APs, I got homework that required thought and problem-solving skills. The teachers "taught to the test" in the sense that they taught the material that was in the AP curriculum- but the range of that curriculum was much wider than the standard New York Regents curriculum that shaped the other college-preparatory courses. They covered most of that range because we didn't know in advance which particulars would be on the test. These courses on my transcript, and my 4 and 5 marks on the exams, helped me get in to a prestigious Name Brand College, and were a sound preparation for the challenges I faced there, including being in classes with students from much more academically-competitive schools, like some of the best schools in the Washington, DC area. My classmates who also went to Name Brand Colleges or the flagship schools of New York or other state university systems also found themselves well-prepared. I hope that my old school continues to offer AP courses and that more students there are encouraged to take them- the ingredients of a higher Challenge Index score. I don't think any one number can summarize a high school's performance, but the Challenge Index is at least as legitimate a measure as average SAT scores or scores on state exams.

Posted by: OverEducated | December 8, 2006 6:39 PM

Hm. I agree with Jay about part of this. I took maybe four genuinely great classes in my Fairfax County high school and two of them were AP classes (and the other two were taught by teachers who also taught AP). Those classes were challenging and interesting and FUN, and they pushed me harder than any other classes I took at that school. I wish I had been able to take more. Great and challenging Shakespeare class? Hahahahahaha, yeah, that would have been nice -- in Bizarro South Lakes.

But...while the teachers were a HUGE and absolutely necessary part of why the classes were good, the other students were an important part too. AP English and calculus were the only classes I ever took in high school that didn't get slowed down for the lowest common denominator. In those two classes--and only those two--students were expected to meet the teacher's pace, and when about 25% didn't, they were placed in a non-AP class after the first quarter. And the remaining quarters were vastly better for it, because we were actually being challenged, by our teachers and by the other students. I had to really work in those classes. I actually -needed- to do my calculus homework before I really nailed down the concepts. I loved it. And no way would I have been ready for the demands of my college professors without the cold shower of a couple genuinely demanding college-level AP classes.

Posted by: Sheila | December 12, 2006 5:55 PM

The comments to this entry are closed.


© 2010 The Washington Post Company