Network News

X My Profile
View More Activity

Debate over school data in wealthy counties

Educational statistics expert Joseph Hawkins, one of my guides to the mysteries of test assessment, is impatient with the way the Montgomery County Public School system, as he puts it, “is always telling the world how better it is than everyone else.” He finds flaws in its latest celebration of college success by county graduates, particularly minorities.

As a senior study director with the Rockville-based research firm Westat, Hawkins’ critique has regional and national importance because it deals with the National Student Clearinghouse. This little-known information source may become the way school raters like me decide which school families and taxpayers are getting their money’s worth and which aren’t.

The clearinghouse has a database of more than 93 million students in more than 3,300 colleges and universities. It originally specialized in verifying student enrollment for loan companies. Now it tells high schools how their alums are doing.

Yeah, sure, says Hawkins, but “data from the Clearinghouse is not completely accurate, especially if social security numbers for students are not obtained.” Also, he says, some of the numbers Montgomery County brags about don’t look so good when compared to others.

“For example,” Hawkins says, “MCPS reports that 26.7 percent of the African American graduates earned a degree in six years. Sounds okay, right? But according to the NCES [National Center for Educational Statistics, part of the U.S. Department of Education] the college graduation rate for black kids is 42 percent. The graduation rate for white kids is 62 percent.” Why, he asks, are MCPS graduates doing worse?

Montgomery County school spokesman Dana Tofig works for Superintendent Jerry D. Weast, the most insatiable consumer and promoter of educational data in America, so Tofig is always ready for statistical queries. He says Hawkins is comparing “apples to pumpkins.” The county is reporting the percentage of black high school graduates from the classes of 2001 to 2004 who got college degrees in six years. The NCES is reporting the percentage of black college freshmen (not including black high school grads who didn’t go to college) who graduated in six years.

It is easy to stumble over these subtle distinctions, which is in part what Hawkins is saying. Montgomery County is one of the best school systems in the country, but has to be candid about the limits of the data supporting the valid argument that families are getting a good deal for the relatively high taxes they pay for these schools.

Washington area office holders---as well as real estate agents---have been extolling the high test scores of our suburban schools for years. They rarely add that communities with such high average incomes nearly always have high scores. It is an iron rule of educational testing, Hawkins’ home ground. We tend to assume a restaurant is good because the customers are well-dressed. In the same way we think a school is good because the parents are affluent. But that is not always the case for all students.

If the National Student Clearinghouse can improve its data-gathering, we would have more indicators of which districts best prepare for college the disadvantaged students Hawkins and I always talk about.

Hawkins onced worked for the Montgomery County schools, and sees their many information-gathering improvements. “I’m glad that MCPS is investing in this data,” he says. But he would like to see more of it, especially comparing the college success of its minority and low-income students to those from other places.

If you think this overlooks kids who don’t go to college, ask what skills good employers and trade schools want in their high school graduate applicants—pretty much the same facility with numbers and words you need to get into a state university. I hope Montgomery County and Hawkins will pursue their frank and friendly discussion to get us even better information on this.

Read Jay's blog every day, and follow all of The Post's Education coverage on Twitter, Facebook and our Education web page.

By Jay Mathews  | July 18, 2010; 6:00 PM ET
Categories:  Metro Monday  | Tags:  Montgomery County's great school data, National Student Clearinghouse, Superintendent Jerry D. Weast, expert Joseph Hawkins says Montgomery can do better  
Save & Share:  Send E-mail   Facebook   Twitter   Digg   Yahoo Buzz   Del.icio.us   StumbleUpon   Technorati   Google Buzz   Previous: The untruth about International Baccalaureate
Next: Rare attack on Harlem Children's Zone

Comments

But if the 26% is of high school grads and the national numbers are higher but are talking about people who are already in college, then that explains the difference and MCPS looks even better.

Posted by: celestun100 | July 18, 2010 8:43 PM | Report abuse

I also don't think Montgomery county taxes are so high, compared to other places in the nation. In the midwest, taxes are way higher.

Posted by: celestun100 | July 18, 2010 8:44 PM | Report abuse

I would agree that MCPS assumption that everyone wants to go to college is a little over the top. Everyone doesn't want to go college.

Posted by: celestun100 | July 18, 2010 8:46 PM | Report abuse

I think what is strangest about their data is the data that schools report on discipline problems. My impression was that they really frowned upon referrals because it made their school data look bad. They never said that directly, it was an impression I had.

Posted by: celestun100 | July 18, 2010 8:49 PM | Report abuse

Maybe Hawkins left MoCo schools in dismay over their annual data gathering from parents, used internally to "evaluate' the schools and the job the administrative and teaching team are doing? [smile]
Response rates to these universally-distributed forms vary from 5% to 15%, so low there's no telling how a representative sample of the population of parents would respond.
I write this because not only would every statistician at Westat know the data are worthless, but because Westat has been the contractor for DCPS for quality-control-checked random sampling of parent satisfaction. Lets start seeing more widespread quality data collection, instead of the feel-good,, report-what-we-like call-in-if-you-love-us data collections schools waste taxpaper money on.

Posted by: incredulous | July 19, 2010 1:22 AM | Report abuse

"There ae three kinds of lies . . . "

Posted by: Nemessis | July 19, 2010 7:24 AM | Report abuse

There are a couple of things that puzzle me about this article. First, I am not sure why an educational statistician such the one cited , Joseph Hawkins, would confuse the MCPS rate of high school graduates who earned a postsecondary degree with the NCES graduation rate for first-time freshmen. While I could see a layman missing the “subtle distinctions”, I do not understand why the expert would.

Second, there seems to be two different messages in the article that are not clearly differentiated. Towards the end of the article, the emphasis seems to be that Montgomery County should be gathering a greater amount of data, especially data concerning minorities and low-income students. However, at other times, the article seems to be a call for a set of indicators that could be used to compare different districts. While the entire article was centered, for good or ill, on the MCPS, the county would not be a primary actor for producing those indicators.

Posted by: Wmcfam | July 19, 2010 12:43 PM | Report abuse

good point, Wmcfam. I really do have to learn to write these things more clearly. I was trying to say both---we need more data and more comparative data.

Posted by: Jay Mathews | July 19, 2010 1:29 PM | Report abuse

If the National Student Clearinghouse can improve its data-gathering, we would have more indicators of which districts best prepare for college the disadvantaged students Hawkins and I always talk about.
.......................................

The idea of the National Student Clearinghouse is flawed since it depends upon the data of students graduating from college. It takes usually four years to graduate.

The results each year only provide information on the public school as it existed four years ago.

In the case of graduation that take five or six years the information on the public school is as it existed for more than four years ago.

The results for a public school as it existed four or more yeas ago can not be used to judge the public school as it now exists since principals change, teachers change, etc.

All that one can say is that according to the data public school A did a good/bad job X years ago.

One of course look at these figures as a pipeline of a moving average where each year there is a new four year figure but this still will be four years out of date and can only tell you the trend four years ago.

The lag of the data is just too great when the retirement of a top notch principal can dramatically affect the performance of a school, just as the replacement of a mediocre principal by a top notch principal can dramatically affect the performance of a school.

An a different principal is only one case of the significant change that can take place in these public high schools in 4 years. Remember this is not Title 1 poverty public schools, but affluent and middle class public schools with parents willing to pay with their property taxes for improvement in the public schools.

In the area of disadvantaged students that are not white racial prejudices have diminished over time. In public schools with the problem of racial prejudices a few new teachers can make a dramatic change.

Stocks that you bought 4 years ago and sold at a healthy profit were great 4 years ago but that does not mean that they are now as great as they were 4 years ago.

I am really surprised how this significant flaw has been missed.

Posted by: bsallamack | July 19, 2010 7:23 PM | Report abuse

Here is a new idea while I wait for Jay Mathews response to the flaw in the data model he thinks is important.

I have extensively reviewed the NCES data regarding national test and discovered that we do not have an achievement gap. Instead we have a free lunch gap since the majority of students that are failing are receiving free lunches and the solution to this problem is to simply not offer free lunches.

Of course this is ridiculous, but is it any less ridiculous than the flawed data as proposed by Mr. Mathews. Is it any less ridiculous in the great concern of measuring the performance of the limited number of disadvantaged students that are in affluent and middle class public schools.

NCES tests for years have indicated that the overwhelming number of disadvantaged students that are failing are in the Title 1 poverty public schools.

Yet no new programs are proposed to deal with the problems of the Title 1 poverty public schools.

Instead there is a pretense that there is a national problem of every public schools in the United States and not specifically the problems of the Title 1 poverty public schools.

Public schools throughout the nation are supposed to spend limited resources while the Federal government simply ignores the problems of the Title 1 poverty public schools.

States are expected to adopt expensive and unnecessary testing and computer systems to evaluate the supposedly effectiveness of teachers from test results of students. Student A failed the test in 5th grade and now fails the test in 6th grade. Supposedly a computer systems now will tell us the effectiveness of teacher AB who was the teacher of student A in the 6th grade.

This is lunacy.

Develop new programs directly aimed at the problems of the Title 1 poverty public schools.

We do not need a plumber fixing a leaky faucet when the toilet is overflowing.

Posted by: bsallamack | July 19, 2010 8:22 PM | Report abuse

MCPS reports that 26.7 percent of the African American graduates earned a degree in six years.

the college graduation rate for black kids is 42 percent. The graduation rate for white kids is 62 percent.”

Why, he asks, are MCPS graduates doing worse?
.....................................
Jay you really need to quote someone better than Joseph Hawkins.

Even from just these figures there is a flaw in the analysis and logic.

All black students who graduate form high school are not alike, just like all white students who graduate from high school.

A little thought would indicate that richer white students will probably do better in college than poorer white students. Look at George W. Bush who graduated from Yale.

Many blacks have moved into the upper middle classes and so there are different economic levels regarding blacks just like there are different economic levels regarding whites. We are far from the days when almost all blacks were poor.

The national college graduation figure will reflect a blending of all the economic classes. You would not be able to use this figure in regard to a specific county since you would not necessarily have the same percentage of economic classes that were blended by the national figure.

Imagine the national figure represents 50 percent of upper middle class blacks. How can you use the national figure in comparison with a county with black graduates that only had a 20 percent of upper middle class blacks.

And this is only one of the various economic levels.

You really need to chose your experts more wisely.

Posted by: bsallamack | July 19, 2010 9:18 PM | Report abuse

The comments to this entry are closed.

 
 
RSS Feed
Subscribe to The Post

© 2010 The Washington Post Company