New evidence that SAT hurts blacks
Roy Freedle is 76 now, with a research psychologist's innate patience. He knows that decades often pass before valid ideas take root. When the notion is as radical as his, that the SAT is racially biased, an even longer wait might be expected. But after 23 years the research he has done on the surprising reaction of black students to hard words versus easy words seems to be gaining new respectability.
Seven years ago, after being discouraged from investigating findings while working for the Educational Testing Service, Freedle published a paper in the Harvard Educational Review that won significant attention.
He was retired from ETS by then. As he expected, his former supervisors dismissed his conclusions. Researchers working for the College Board, which owns the SAT, said the test was not biased. But the then president of the University of California system, a cognitive psychologist named Richard C. Atkinson, was intrigued. He asked the director of research in his office to replicate Freedle's study.
Now, in the latest issue of the Harvard Educational Review, the two scholars who took on that project have published a paper saying Freedle was right about a flaw in the SAT, even in its current form. They say "the SAT, a high-stakes test with significant consequences for the educational opportunities available to young people in the United States, favors one ethnic group over another."
"The confirmation of unfair test results throws into question the validity of the test and, consequently, all decisions based on its results," said Maria Veronica Santelices, now at the Pontificia Universidad Catolica de Chile in Santiago, and Mark Wilson of UC Berkeley. "All admissions decisions based exclusively or predominantly on SAT performance--and therefore access to higher education institutions and subsequent job placement and professional success--appear to be biased against the African American minority group and could be exposed to legal challenge."
Researchers at the College Board and ETS don't like this new paper anymore than they liked Freedle's in 2003. Laurence Bunin, the College Board vice president in charge of the SAT, said the Santelices-Wilson study is "fundamentally flawed." He pointed out that it had not yet been peer reviewed. He said the scholars' conclusions were "wrong and irresponsible and a disservice to students, parents and colleges," and were based on "a very small, limited and unrepresentative sample."
College Board spokeswoman Kathleen Steinberg said the Harvard Educational Review declined the College Board's offer of a response to the paper, but plans to publish a criticism of the paper by ETS researcher Neil Dorans, as well as a response by Freedle himself.
They are discussing a complex topic, full of psychometric terms and concepts I am not competent to judge. Back in 2003, when I wrote a long article for the Atlantic Monthly on Freedle's work, I relied heavily on him and other experts to explain what they were talking about. Much of it had to do with a method of test analysis called differential item functioning, or DIF (rhymes with cliff). Psychometricians like Freedle and his colleagues at ETS, which was then managing the SAT, looked at how different ethnicities that were matched at different scoring levels (those who had scored 360 on the SAT verbal test, then those who had scored 380, and so on) did on each item.
At each level of ability, but particularly in the lower-scoring groups, white students on average did better than blacks on the easier items, whereas blacks on average did better than whites on the harder ones. (Whites, however, as a group did better overall.)
This was unexpected. The deeper Freedle got into it, the more uncomfortable his supervisors seemed to be with his work. He had to revise one paper more than 11 times before they allowed him to publish it.
Hard questions, those that produced more wrong answers, tended to have longer, less common words. Easy questions tended to have shorter, more common words. Freedle thought this was key to the relative success African American students had with the harder ones. Simpler words tended to have more meanings, and in some cases different meanings in white middle class neighborhoods than they had in underprivileged minority neighborhoods, he concluded. This, he said, could help explain why African American students did worse on questions with common words than on questions that depended on harder, but less ambiguous words they studied at school.
On average, he said, black students were performing only slightly above matched-ability whites on hard questions. But averages did not submit applications to colleges. Individual students did. Some of those individuals, he discovered, would have gotten a boost of a hundred points or more on the SAT if the score was weighted toward the hard items. He proposed that the College Board offer a supplement to SAT scores, called the Revised-SAT, or R-SAT, which would be calculated based only on the hard items. This, he said, would "greatly increase the number of high-scoring minority individuals."
In their paper, Santelices and Wilson rule out Freedle's suggestion that the bias he found in the test might affect all kinds of multiple-choice questions, or minorities other than blacks. But they did find it in sentence completion and reading comprehension sections of the SAT.
Saul Geiser was the director of research in Atkinson's office originally given the assignment to look into Freedle's theory. Eventually he arranged for Santelices, then a doctoral candidate at UC Berkeley, to do the research as her PhD thesis, working with Wilson, a UC Berkeley psychometrician who had also been asked to look at Freedle's work.
Geiser said he thinks the two researchers did a good job. He does not agree with Bunin's criticisms of their work. He said he, like Freedle, wants more more research on why blacks and whites answer these questions differently, so that any unfair disadvantages for blacks can be removed.
He said he thought the College Board, in particular, should "get over the denial" of any merit to what Freedle has discovered. That may take a while. The College Board, after all, may be right that the SAT is unbiased.
But the new paper means more researchers are likely to go more deeply into what Freedle has found, and eventually settle the question of what should be done about it.
Read Jay's blog every day at http://washingtonpost.com/class-struggle.
Follow all the Post's Education coverage on Twitter, Facebook and our Education web page, http://washingtonpost.com/education.
| June 17, 2010; 10:00 PM ET
Categories: Trends | Tags: College Board denies bias, Maria Veronica Santelices, Mark Wilson, Roy Freedle, blacks do better on harder questions, racial bias in the SAT
Save & Share: Previous: Why mom disliked my summer reading
Next: Hispanic students taking AP Spanish: a scam?
Posted by: tomsing | June 18, 2010 12:12 AM | Report abuse
Posted by: Cal_Lanier | June 18, 2010 2:29 AM | Report abuse
Posted by: Denny4 | June 18, 2010 5:36 AM | Report abuse
Posted by: johnt4853 | June 18, 2010 9:37 AM | Report abuse
Posted by: afsljafweljkjlfe | June 18, 2010 10:36 AM | Report abuse
Posted by: jsump | June 18, 2010 1:08 PM | Report abuse
Posted by: jsump | June 18, 2010 1:27 PM | Report abuse
Posted by: CrimsonWife | June 18, 2010 1:56 PM | Report abuse
Posted by: viaexports | June 18, 2010 2:15 PM | Report abuse
Posted by: fairfaxvaguy | June 18, 2010 2:57 PM | Report abuse
Posted by: Jay Mathews | June 18, 2010 4:54 PM | Report abuse
Posted by: Trev1 | June 18, 2010 11:30 PM | Report abuse
Posted by: tomsing | June 19, 2010 12:03 AM | Report abuse
Posted by: tomsing | June 19, 2010 7:26 AM | Report abuse
Posted by: FairTest | June 19, 2010 9:11 AM | Report abuse
Posted by: fairfaxvaguy | June 19, 2010 10:37 AM | Report abuse
Posted by: Cal_Lanier | June 19, 2010 6:26 PM | Report abuse
Posted by: steve10c | June 19, 2010 6:29 PM | Report abuse
Posted by: Trev1 | June 19, 2010 8:36 PM | Report abuse
Posted by: Cal_Lanier | June 19, 2010 9:35 PM | Report abuse
Posted by: DamarisS | June 20, 2010 10:42 AM | Report abuse
Posted by: chrisroman | June 20, 2010 11:31 AM | Report abuse
Posted by: linsee1 | June 20, 2010 11:39 AM | Report abuse
Posted by: eaglechik | June 21, 2010 1:02 PM | Report abuse
Posted by: dhartmanva | June 21, 2010 1:04 PM | Report abuse
Posted by: eaglechik | June 21, 2010 1:12 PM | Report abuse
Posted by: Cal_Lanier | June 21, 2010 2:10 PM | Report abuse
Posted by: mental69 | June 21, 2010 3:09 PM | Report abuse
Posted by: eaglechik | June 21, 2010 3:12 PM | Report abuse
Posted by: junksciencemom | June 21, 2010 5:09 PM | Report abuse
Posted by: Cal_Lanier | June 21, 2010 5:50 PM | Report abuse
Posted by: Jay Mathews | June 21, 2010 7:24 PM | Report abuse
Posted by: seraphina21 | June 21, 2010 8:46 PM | Report abuse
Posted by: JesseF | June 21, 2010 11:41 PM | Report abuse
Posted by: FYIColumbiaMD | June 22, 2010 7:54 AM | Report abuse
Posted by: telecomprof | June 22, 2010 10:55 AM | Report abuse
Posted by: incredulous | June 22, 2010 2:53 PM | Report abuse
Posted by: Gonzage1 | June 22, 2010 3:43 PM | Report abuse
Posted by: incredulous | June 23, 2010 9:36 AM | Report abuse
Posted by: momof4md | June 23, 2010 10:21 AM | Report abuse
Posted by: Bob017 | June 23, 2010 9:55 PM | Report abuse
Posted by: Trev1 | June 24, 2010 6:25 PM | Report abuse
Posted by: doglover6 | June 24, 2010 7:24 PM | Report abuse
The comments to this entry are closed.