Test that makes U.S. look bad may not be so good
Politicians and pundits are using results from the Programme for International Student Assessment|(PISA) tests to say our kids are falling behind the rest of the world, so maybe we should get some PISA practice. Brookings Institution scholar Tom Loveless, a member of the U.S. advisory board to PISA, offered this sample question for 15 year olds from the mathematics literacy section of the exam:
For a rock concert a rectangular field of size 100 m by 50 m was reserved for the audience. The concert was completely sold out and the field was full with all the fans standing. Which one of the following is likely to be the best estimate of the total number of people attending the concert?
I think this is a bad question, and not just because I got it wrong. I said 5,000 The answer is 20,000. I don’t see why deciding four people, not one, would fit better in a square meter is a sign of math literacy. There are some people I don't like to get close to at concerts.
Loveless, an expert on international testing, agrees that the problem was ill-chosen. “I think it would throw kids off,” he said. “The math is rather trivial.” There are other problems with PISA, such as an ideological bias and a tendency to assume cause-and-effect relationships. But the American results on those exams — we are below average in both math and science literacy — are often cited as a national disgrace.
A key failing of PISA, Loveless said, is “it does not measure what kids have learned in school.” Why not? Because PISA exams are written by the losing side in a century-old debate over how to teach math. For convenience, call the pro-PISA people progressives and the anti-PISA people traditionalists. Loveless, who taught sixth grade in California for nine years before getting his Ph.D. in education from the University of Chicago, is a traditionalist but appreciates the arguments on both sides. The progressives want to make math instruction more relevant to the real world, and emphasize mathematical reasoning more than calculation. The traditionalists say you can’t reason well without mastering the fundamentals. They dislike their approach being dismissed by progressives as “shopkeeper math,” Loveless said, “like it was old-fashioned to try to compute anything.”
Loveless prefers the other major international test, Trends in International Mathematics and Science Study, which aligns with the way U.S. students are taught. He is the U.S. representative to the General Assembly of the International Association for the Evaluation of Educational Assessment, a 60-nation body that oversees TIMSS.
Policy makers are right to worry about our world standing in math and reading. But their interpretations of the data are often flawed, and make our schools look worse than they are. The Wall Street Journal headlined a 2004 PISA report: “U.S. Teens Are Among Worst at Math.” But on TIMSS we are above the international average in both 4th grade and 8th grade math.
Many PISA questions are refreshingly challenging, but conservative politicians who praise the test's rigors should be careful, since the attitudinal parts of PISA lean left. On PISA’s student questionnaire, those who support statements such as “I am in favor of having laws that regulate factory emissions even if this would increase the price of products” are deemed to be environmentally responsible. Those who disagree are not.
This is not to say international comparisons are useless. We can learn from the differences between us and the countries at the top of the TIMSS (and in some cases also the PISA) list —Hong Kong, Korea, Singapore, Taiwan and Japan. Their families tend to devote more out-of-school time to learning. Their curricula and textbooks are leaner and more focused. Their teachers often train in academic disciplines, not at education schools.
In those countries, hard-working students are admired by their peers. Here they are often dismissed as freakish nerds. “America has always been ambivalent about how academics fit in our culture,” Loveless said.
That’s okay with me. I think our economy, politics and society have benefited from having as much respect for street smarts as book smarts. Being unimpressed with academic degrees saves us from the stultifying aspects of European-style class divisions. But we pay a price. Perhaps it would help if we found ways to celebrate our grade school math whizzes as much as our playground stars. We want to use the right tests, but more hoorahs, even from 15 year olds, for people who do their homework can’t hurt.
Posted by: researcher2 | October 19, 2009 7:04 AM | Report abuse
Posted by: ClausvonZastrow | October 19, 2009 8:16 AM | Report abuse
Posted by: dlnewquist | October 19, 2009 12:25 PM | Report abuse
Posted by: flyfisher_20750 | October 19, 2009 12:32 PM | Report abuse
Posted by: sscritic | October 19, 2009 12:38 PM | Report abuse
Posted by: FairlingtonBlade | October 19, 2009 1:11 PM | Report abuse
Posted by: mytwocents | October 19, 2009 1:36 PM | Report abuse
Posted by: doglover6 | October 19, 2009 1:47 PM | Report abuse
Posted by: speakuplouder | October 19, 2009 1:56 PM | Report abuse
Posted by: bobtom222 | October 19, 2009 3:18 PM | Report abuse
Posted by: CrimsonWife | October 19, 2009 4:07 PM | Report abuse
Posted by: someguy100 | October 20, 2009 9:40 AM | Report abuse
Posted by: Jay Mathews | October 20, 2009 2:06 PM | Report abuse
Posted by: wswilson1 | October 20, 2009 9:23 PM | Report abuse
Posted by: TCHatch | October 22, 2009 5:47 AM | Report abuse
The comments to this entry are closed.