The lowdown on standardized tests and how they are scored
You wouldn’t be alone if you thought that the companies that score standardized tests have machines that can whip out the results in no time.
But they don’t, at least not for tests with questions that require a written response.
Instead, they hire tens of thousands of temporary workers, who are paid by the hour (often about $11 to $13 per), to decipher the answers of millions of kids from third to twelfth grade. They are often poorly trained and overworked.
That’s the assessment from one of those workers, Dan DiMaggio, who explains in detail his experiences in an article called “The Loneliness of the Long-Distance Test Scorer,” which you can find here.
His conclusion: “Why would people in their right minds want to leave educational assessment in the hands of poorly trained, overworked, low-paid temps, working for companies interested only in cranking out acceptable numbers and improving their bottom line? Though the odds might seem slim, our collective goal, as students, teachers, parents—and even test scorers—should be to liberate education from this farcical numbers game."
In the article, DiMaggio refers to a book by Todd Farley called “Making the Grades: My Misadventures in the Standardized Testing Industry,” an indictment of the industry based on Farley’s 14 years in the industry.
Here’s a new, somewhat satirical piece about standardized tests and scoring by Farley, first published on The Huffington Post.
By Todd Farley
Like the maddeningly successful author Diane Ravitch, I, too, have changed my mind about No Child Left Behind. Unlike the estimable Ravitch, however -- whose recent bestseller argues in exhausting detail against the very accountability measures that Ravitch long championed -- in the great testing debate I’ve gone from "con" to "pro."
Since 1994, when I first got hired as a lowly temp for measly wages to spend mere seconds glancing at and scoring standardized tests, until the release of my non-bestselling book last fall, I had steadfastly believed that large-scale assessment was a lame measure of student learning that really only benefited the multinational corporations paid millions upon millions upon millions of dollars to write and score the tests.
I began to see the error of my ways last Thanksgiving, however, just as soon as my huge son popped from his mother’s womb, keening and wailing, demanding massive amounts of food, a closet full of clothing, and the assistance of various costly household staff (baby-sitter, music teacher, test-prep tutor, etc.). Only then, as my little boy first began his mantra of "more, more, more," did I finally see standardized testing for what it really is: a growth industry. In these times of economic recession, it was a lesson I didn’t need to learn twice.
Since that educational epiphany, the benefits of standardized testing have become embarrassingly obvious to me, starting with the fact the industry has proven to be a jobs program virtually unmatched since FDR’s WPA.
There’s pretty much no one who can’t get a job in testing, whether it’s as one of the tens of thousands of temps hired each year to score student responses to tests or as one of the teacher/ex-teacher/once-knew-a-teachers hired to write them. Because of the massive influx of money swamping the testing industry due to President Obama’s Race to the Top, anyone who taught school or went to school or even drove past a school is eligible for work in the business.
Consider that in the last year I’ve seen people hired back to testing companies who had been run off in shame not long before.
I’ve watched people fired from one testing company immediately getting rehired by another.
I’ve witnessed tiny, mom & pop test-development vendors celebrating their first contract by immediately posting job listings on Craigslist, hoping to find someone, anyone, out there on the great, big Internets that might be able to help them write "rigorous," national tests.
Even me -- a guy who a year ago was spouting virulent anti-testing rhetoric on NPR and whose "down-with-standardized-testing!" editorials were gracing the pages of The New York Times, Christian Science Monitor, Education Week, etc... has been offered absolution, and today I earn fat paychecks from more than one of the industry’s stalwart companies. (No, I have no shame.)
Forgiveness, obviously, is one of the industry’s strengths, and once hired testing industry employees and companies both enjoy the sort of tenure that would make a teachers’ union proud -- those jobs and those contracts pretty much can’t be lost.
For instance, in the last year I’ve seen a test development company fired for poor quality work nearly immediately rehired with a contract four times as large.
I’ve watched multinational corporations like Pearson Education get dressed down in U.S. Department of Education audits and fined by their customers for shoddy work and then be awarded contracts worth hundreds of millions of dollars.
I’ve witnessed a state like Tennessee win a half a billion dollars in federal education money even after another audit pointed out scoring inequities on that state’s tests. Yup, the testing industry today is like that Cole Porter song "Anything Goes," which is just what an economy as troubled as our desperately needs.
Standardized testing’s other most redeeming characteristic may be a surprising one, but the industry has proven to be really good at recycling.
Just as an employee found wanting at one company can simply get a job at another (or can start his/her own lucrative consulting company), so can test questions and/or whole tests be used again and again and again. Test questions good enough to be used on a Chicago test can be used again on a New York City test, just as a complete test sold for use in California can be sold again for use in Texas.
It’s happening even with the exalted Common Core Standards, or CCS, those educational benchmarks that many people (Messieurs Obama and Duncan included) believe will save this country’s sorry educational system: Just the other day I saw a testing company advertising for people on Craigslist to re-align the company’s millions of existing test questions to those almighty new standards.
Even though those millions of test questions in the company’s database had been written to apparently crappy and definitely passé state standards, that innovative testing company was making them magically new simply by draping them in new clothes, linking those items -- absolutely unchanged -- to the CCS. That clever company was saving an immeasurable amount of time and effort by recycling those old items instead of writing new ones, just the sort of reuse of materials that I’m sure would make Al Gore proud (not to mention Gore’s minion, Davis Guggenheim).
While my many previous concerns about the efficacy of standardized testing have not gone away, the primacy of those concerns has been supplanted by the fact my son has an enormous appetite. In my view, today the standardized testing industry is like one of those phone booths filled with whirling Race to the Top millions, and the only smart thing for me to do as a husband and father is to grab as much as I can. Like Diane Ravitch, I’ve seen how wrong I used to be.
I am reformer! Hear me roar.
Follow my blog every day by bookmarking washingtonpost.com/answersheet. And for admissions advice, college news and links to campus papers, please check out our Higher Education page at washingtonpost.com/higher-ed Bookmark it!
| December 22, 2010; 5:00 AM ET
Categories: Guest Bloggers, Standardized Tests | Tags: common core standards, how tests are scored, no child left behind, race to the top, standardized test industry, standardized test scoring, standardized tests, test industry, test scoring
Save & Share: Previous: The history of Rudolph the Red-Nosed Reindeer
Next: Bring back shop class
Posted by: pdexiii | December 22, 2010 6:25 AM | Report abuse
Posted by: jbeeler | December 22, 2010 7:44 AM | Report abuse
Posted by: barbarachina | December 22, 2010 8:43 AM | Report abuse
Posted by: Cal_Lanier | December 22, 2010 9:26 AM | Report abuse
Posted by: buckbuck11 | December 22, 2010 10:54 AM | Report abuse
Posted by: kmlisle | December 22, 2010 11:23 AM | Report abuse
Posted by: rtinindiana | December 22, 2010 12:29 PM | Report abuse
Posted by: jlp19 | December 22, 2010 1:35 PM | Report abuse
Posted by: jlp19 | December 22, 2010 1:37 PM | Report abuse
Posted by: Trev1 | December 22, 2010 10:21 PM | Report abuse
Posted by: aed3 | December 23, 2010 10:45 AM | Report abuse
Posted by: DisgruntledTeacher | December 23, 2010 5:40 PM | Report abuse