Network News

X My Profile
View More Activity

Posted at 10:00 AM ET, 12/30/2009

The value of standardized tests

By Valerie Strauss

I think I’ve left a false impression.

Three times in the past month I’ve had people tell me that I am anti-test. They have gotten this idea from my posts about the dangers of high-stakes standardized testing.

One was a story about how I was diagnosed with “over comprehension” in second grade because I gave more than one answer on a multiple choice standardized test and annotated them in the margins.

For whatever record there is, I am not opposed to tests. It seems painfully obvious that some tests are important and valid--and if tests didn’t exist, we’d go ahead and create them. What is troubling is the way we use the results of tests to judge students, schools and teachers--even when these tests are poorly designed or tell us little about true ability.

I received the following email from Patrick Mattimore, an attorney who formerly taught Advanced Placement Psychology in San Francisco and who is a fellow at the non-profit Institute for Analytic Journalism. He has written critically about the AP program and standardized tests, such as the SAT and the SAT Subject tests.

Mattimore just moved to Beijing two weeks ago, from France, and is assisting his wife, Jean Amabile, who works for an non-governmental organization that trains defense attorneys throughout the world to represent defendants.

Here’s what he wrote to me. (I removed the repeated titles of officials.) Tell me what you think in the comments or at .

I disagree with your position re. standardized multiple choice tests. Here’s why.

At the National Education Association Representative Assembly in 2007, Democratic presidential contenders routinely lambasted the “No Child Left Behind” law, focusing largely on the testing requirements of the law.

Then candidates President Obama, Vice President [Joe] Biden, and Secretary of State [Hillary] Clinton all referred disparagingly to children filling in bubbles on standardized tests. Clinton and Biden suggested that children were being forced to memorize things which, according to Clinton, got in the way of learning and, to Biden’s way of thinking, subverted critical thought. Clinton further opined that we were allowing our testing to drive our curriculum, rather than the other way around.

Obama said that teachers had devoted themselves to teaching, not testing. Former Sen. John Edwards pointed to the tee shirts that were being distributed at the convention and repeated the shirt’s message that a child was more than a test.

These sentiments all make sense but they are fundamentally flawed.

The politicians’ ideas imply an either/or situation when, in fact, standardized tests should go hand-in-hand with teaching. Testing and teaching are and must be conjoined. Each informs the other. Memorization is not inimical to either learning or critical thinking but is often the touchstone for those higher-order skills.

To many educators, standardized multiple choice tests are the wrong way to assess what our kids are learning. But those tests are and should be the primary instrument we use to measure student progress.

Multiple choice tests are everywhere in America. They are the primary yardsticks in our K-12 assessments. Among other things, we use multiple choice tests for: college admissions; graduate school admissions; teacher licensing programs; licensing drivers, and; admission to professional practices, such as state bars.

Multiple choice tests provide fast results, allowing teachers to provide immediate feedback and corrections. They are accurate, easy to administer and understand, objective, can be norm or criterion referenced, and most importantly, can test a variety of complexities of student knowledge.

To suggest that filling in bubbles is an endpoint of instruction misses the obvious point that the bubbling activity is a means for checking how well students are learning. Critics claim that the tests stifle creativity by forcing students to think in terms of right answers instead of possibilities, unfairly brand students, and measure only narrow ranges of ability.

But, a well-constructed multiple choice test will not only measure a student’s retention of facts, but test that student’s ability to apply what she has learned to novel problems and to make connections and inferences. A multiple choice test that incorporates a taxonomy of higher levels of thinking will force students to analyze, evaluate, and synthesize information.

Standardizing the tests insures that our students are being challenged on like measures and enables us to see across the board what is working and what is not. As an assessment measure, standardized multiple choice tests may not paint a student’s complete academic portrait but they certainly provide us with a reasonable snapshot.

Secretary of Education Arne Duncan should insist upon standardized national assessment measures for all our students in every grade and those assessments should be primarily multiple choice.

Our elected representatives should disregard the educational reform chant that suggests that our current classroom deficiencies spring from the primary measures that help us diagnose our students’ shortcomings. This is a classic case of shooting the messenger when what we should be doing is looking to that messenger to assist us in solving our educational problems.

Follow Valerie’s blog all day, every day at

For all the Post’s Education coverage, please see

By Valerie Strauss  | December 30, 2009; 10:00 AM ET
Categories:  Standardized Tests  | Tags:  standardized tests  
Save & Share:  Send E-mail   Facebook   Twitter   Digg   Yahoo Buzz   StumbleUpon   Technorati   Google Buzz   Previous: Jay Mathews, E.D. Hirsch and other education greats honored
Next: Obama's view of sex education broader than Bush's


Without tests, the public is asked to provide money without any controls on whether it is spent to achieve the stated goals.

What other substitute could exist other than "just so" stories by the educators, the recipients of the public monies, who have a financial interest in reporting that they are doing a proper job.

Posted by: edbyronadams | December 30, 2009 10:49 AM | Report abuse

Mattimore's analysis is spot-on.

I currently teach both "regular" and AP-level classes and I worked in the testing industry in a previous job. I regularly hear the complaints listed in Mattimore's email from teachers, administrators, parents, and students.

Basic facts (and not-so-basic facts) are building blocks. In order to engage in higher-levels of learning, students must have a firm grasp of basic facts.

Many of my students ask for more "debates" and "discussions" in class and want a larger portion of their grades based on participation. However, the students who make these requests are usually those with the worst grasp of the most basic facts.

This phenomenon partially explains the grade inflation that is rampant in our schools. Grades rarely reflect students' knowledge and retention of the actual academic content. Instead, grades are based on students' willingness to look up answers in a book, complete "projects" that allow students to spit back the words they heard in class or read in a book, and how often they speak in class. This is why standardized tests are absolutely essential--they provide an outside measure of what students actually know.

I'd also like to add that I see a connection between the political discourse in this country and the devaluation of "facts" in our schools. Everyone seems to have strong opinions on both sides of the political spectrum, but few actually understand the issues they are discussing (just look at the comments posted on this website). Just once I'd like to hear someone say "I don't really know enough about the topic to have a valid opinion".

Posted by: EnricoPolatzo | December 30, 2009 11:25 AM | Report abuse

Clearly, Mattimore is not a teacher. There are reasons why many teachers are opposed to high-stakes testing - reasons that have nothing to do with their salaries or
reputations, or with teacher unions. Here are a few of them:

1. Tests are not as accurate as Mattimore makes them out to be. Many students in Title I schools must take periodic benchmark multiple-choice tests. These tests are administered so frequently that, by the time April rolls around, many students have testing-ennui. I was once praised for my progress with a very difficult student whom I saw make random patterns on his bubble sheet without looking at the test booklet. Similarly, I have seen normally high-performing students get so burnt out by April, they speed through their tests so that they can read the books under their chairs.

2. Many psychometricians argue that multiple-choice test prep workbooks undermine the validity of the testing measure. Nevertheless, taxpayer dollars are spent on workbooks like Holt's TAKS Test Prep and Spectrum's FCAT Test Prep. Many teachers, especially in Title I schools where the stakes are highest, do nothing but test prep skill-drill in the month leading up to the test. Which brings me to my next point...

3. The presence of high stakes testing in the American classroom changes the nature of learning. Many Title I schools given the label "corrective action" by NCLB are willing to do whatever it takes to make AYP. This means dropping social studies and science from the curriculum, teaching reading and math through test preparation workbooks so that students are intimately familiar with the format of the test and with testing strategies, and focusing most on a middle group of students that have the best chance of moving into the next-highest proficiency bracket (while ignoring the needs of the high and low achievers). Again, psychometricians argue that these strategies render the testing measures invalid.

I hope that Mattimore will not find my observations alarmist or overstated. There is a wide body of thoughtful, balanced research literature documenting experiences similar to mine. For starters, please see Hursch's 2005 study, "The Growth of High Stakes Testing in the USA," and studies conducted by Mary Lee Smith in - yes - the early 1990s.

A final note: I would be happy to have once-a-year standardized testing, provided that there was more regulation associated with them. It is very inappropriate for schools to spend 8 hours per day exclusively on reading and math when the state standards indicate that social studies and science should be taught. Such practices, along with test preparation workbooks which skew the validity of the assessment tool, should be banned. (These regulatory measures would be highly unpopular with textbook companies, but it's safe to say that Title I teachers and students would be relieved.)

Posted by: -JP- | December 30, 2009 12:37 PM | Report abuse

Mattimore grossly overstates the accuracy and objectivity of standardized tests, and offers little to back up such claims. Such tests can measure retention of facts and skills to a degree, and they can be useful for measuring a student's analytical ability to a degree, but real life doesn't give students four or five options, of which two are obviously wrong, one is probably wrong, and two seem like they could be right.

My own experience on the ACT offers a good example of how standardized tests can give false impressions. When I took the ACT, I'd yet to have any trigonometry in my math classes. As a result, the last ten or fifteen questions in the math section were beyond my knowledge. At best, I could make educated guesses, but I am pretty good at screening out absurdly wrong answers, and basic geometry gave me a few hints. So I made my guesses, and I ended up with more than 50 percent correct in that section, despite having done none of the actual math. Someone reading these scores would get the wrong impression about what I had and had not learned. Does that trig section show that I'm a good guesser, or that I've had trig but haven't mastered it yet, or what? How is a college admissions officer supposed to know precisely what has been taught in my high school math class at the precise moment when I took this test?

The fact is that we aren't really testing students' retention abilities on a lot of these national standardized tests when students at some schools with lots of AP courses have had the chance to leap ahead and learn the material while students at poorer inner-city and rural schools may not even have seen the material yet at all. We don't have a national standardized curriculum that puts every student into the same core classes that teach the same material at the same time during the school year. Countries like Japan do have this, and their testing is somewhat more meaningful as a result. But in the United States, we have a patchwork of different tracks in our educational system, with variations state-to-state and among rural, urban, and sub-urban schools, and all of this makes testing a fairly futile effort.

This is not to say that standardized testing does not have its place, because it does. However, such scores need to be put carefully into context, and too often when dealing with thousands of students, they are not. Meanwhile, the testing industry, aided by hyper-anxious parents and students and by colleges that require test scores, promotes the value of such tests and makes a lot of money in the process for a service that probably isn't all that valuable in getting a student into college in the first place. Yes, the scores can be useful, but were I an admissions officer, I'd put the scores at the bottom of the application and probably already have arrived at a decision before looking at them.

Posted by: blert | December 30, 2009 1:17 PM | Report abuse

Why can't standardized tests be one of many assessment tools used to measure student learning? A writing test could be employed (of the sort used on the SAT) to assess students’ ability to form and evaluate arguments as well as to provide short responses to factual questions. Additionally, electronic portfolios that capture students' work in the classroom could be assessed by an independent source. It is also likely that we can improve standardized tests so that they provide a better view of what students know and have learned.

So, I would not give up on standardized tests; they do provide valuable feedback. My recommendations would be to develop additional forms of assessment and to continue to work on improving how well standardized tests measure learning.

Posted by: chmay | December 30, 2009 2:37 PM | Report abuse

Independent researchers have found that standardized tests are far less accurate and fair measures of educational performance than Patrick Mattimore assumes. More importantly, Mattimore writes as if these exams are properly used when many high-stakes applications (most often those mandated by politicians) violate the standards of the measurement profession and often the guidelines posted by the test-makers themselves.

For data and citations, check out the many free fact sheets available at
One good place to start would be:

Posted by: FairTest | December 30, 2009 2:53 PM | Report abuse

Secretary of Education Arne Duncan should insist upon standardized national assessment measures for all our students in every grade and those assessments should be primarily multiple choice.
Just a political game with the 4 billion of Race To The Top going to different state "standarized" tests of "test them until they drop" and "teach to the test", instead of national standardized tests.

Apparently knowing 2+2=4 is different in each state and the question whether the word "dollar" is money has to take into account which state you are in.

It appears that the only thing that is the same in each state is politics.

Posted by: bsallamack | December 30, 2009 4:02 PM | Report abuse

I agree with -JP- and would add that of course teachers are going to "teach to the test" when so much rides on the outcome of that test. There are serious consequences for schools if they don't make adequate yearly progress, and the state doesn't care if a student has a bad day or just isn't developmentally ready for algebra. If tests that count (not district benchmarks) were given multiple times per year, or if other factors that determine a school's performance were measured, there wouldn't be such pressure to teach to that one test. It is absolutely a crime that because science and social studies aren't used to determine yearly progress that they are being excluded from the curriculum.

To blert - I know I've seen research that shows that when a school district starts using a new test, its scores go down for a year or two until the teachers see enough tests to know how to teach to it. If the tests were just testing content knowledge, the scores shouldn't change like that.

To chmay - California had a writing test for 4th and 7th graders, but this year they had to cancel it because the state can't afford to hire people to grade it. The reason almost all tests are multiple choice is that they can be graded cheaply by machines.

Posted by: landerk1 | December 30, 2009 7:18 PM | Report abuse

The testing “obsession” is a snake oil industry, no different than the TV channels ripping off people selling cheap watches on Shop NBC or the Shamwow huckster selling miracle towels for $19.95.

Arne Duncan, Harry Wong, Robert Marzano, and other “educational leaders” from the achievement / assessment test industry will go down in public educational history as villains, frauds and corruptors whose true interest were to acquire power and influence so as to operate get rich quick schemes, deceptively based on helping children learn.

Unhappily, their curriculum narrowing, force fed, one size fits all, system of learning has proved to be cruel, boring, lazy, and of little use to students trying to find enthusiasm for education. In addition, this style of learning has not prepared students to find worthwhile employment in the real world.

These “test nuts” sell systems and structures, sold through commercial web sites, video tapes and “superstar appearances” simply to justify a top down system of management aimed at destroying individualism in teaching while breaking down education policy into an incomprehensible vocabulary of gibberish theory and actions plans based on the “all knowing, all powerful, and the ‘too smart to question leader’” who bases his administration on raising test scores that mean little to a world outside the testing industry, but keep educational numbers theorists and school administrators employed and in power during a recession.

The result of their “destroying the public education experience” will be the abandonment of the physical plant we call the public school system as learners seek a broader and richer educational experience online, and at a much cheaper cost to the taxpayer.

Posted by: noscott | December 31, 2009 12:05 AM | Report abuse

Those testing companies must be making lots of money.

Posted by: aby1 | December 31, 2009 6:18 AM | Report abuse

The comments to this entry are closed.


© 2010 The Washington Post Company