Network News

X My Profile
View More Activity


Posted at 11:59 AM ET, 07/19/2010

New test scoring problems for Pearson

By Valerie Strauss

On the heels of a major problem with the scoring of standardized tests in Florida, NCS Pearson finds itself with another scoring mess, this time in Minnesota.

According to the Pioneer Press, results for Minnesota’s science standardized test aren’t being released on time because Pearson incorrectly scored two questions on the fifth- and eighth-grade tests. The delay is expected to be at least a week, which would be far shorter than the recent delay in Florida.

The Florida delay has become a huge problem for Pearson. First there was a months-long delay in scores of the Florida Comprehensive Assessment Test, earning the company at least $3 million in fines. Once delivered, the scores have been challenged by more than half of the districts in the state.

The Florida Department of Education has a list of all of the individual complaints from the different district, which you can see here.

Following is a history of problems Pearson has encountered with scoring of standardized tests. The concern isn’t primarily that delays and mistakes occur but that school districts continue to use standardized test results for high-stakes purposes.

Remember that these are the errors that we know about. Surely there are plenty we never uncover, making the high-stakes use of these scores unacceptable.

This isn’t just my opinion. As Bob Schaeffer, public education director of the nonprofit National Center for Fair & Open Testing, or FairTest, notes, it is in the Standards for Educational and Psychological Testing distributed by the American Educational Research Association, the National Council on Measurement in Education, the American Psychological Association and other organizations.

Here's the list, compiled by Schaeffer:

1998 -- California – Test score delivery was delayed.

1999-2000 -- Arizona – Twelve thousand tests were misgraded due to flawed answer key.

2000 Florida – Tst score delivery delayed resulting in $4 million fine.

2000 --Minnesota – Misgraded 45,739 graduation tests sparked a lawsuit with an $11 million settlement. A judge found "years of quality control problems" and a "culture emphasizing profitability and cost-cutting." (Note: FairTest consulted with plaintiffs’ attorneys.)

2000 -- Washington – 204,000 writing Washington Assessment of Student Learning exams rescored

2005 -- Michigan -- scores delayed and fines levied per contract

2005 -- Virginia -- Computerized test was misgraded and five students were awarded $5,000 scholarships.

2005-2006 -- SAT college admissions test – 4,400 tests wrongly scored; $3 million settlement after lawsuit (note FairTest was an expert witness for plaintiffs)

2008 -- South Carolina --“Scoring Error Delays School Report Cards,” The State, November 14, 2008

2008-2009 -- Arkansas -- First graders were forced to retake exam because real test was used for practice.

2009-2010 -- Wyoming – New computer adaptive Proficiency Assessments for Wyoming Students flopped; state ordered Pearson to repay $9.5 million for “complete default of the contract.”

2010 Florida – Test score delivery was delayed by more than a month and fines were imposed. More than half of the superintendents in the state questioned score accuracy.

2010 Minnesota -- Results from online science tests taken by 180,000 students were delayed due to scoring error.

-0-

Follow my blog all day, every day by bookmarking washingtonpost.com/answersheet. And for admissions advice, college news and links to campus papers, please check out our Higher Education page at washingtonpost.com/higher-ed Bookmark it!



By Valerie Strauss  | July 19, 2010; 11:59 AM ET
Categories:  Standardized Tests  | Tags:  FCAT results delayed, FCAT scores delayed, FairTEST, fairtest, minnesota test scores delayed, ncs pearson, pearson and FCAT, pearson and minnesota tests, pearson and scoring tests, standardized tests, test results delayed  
Save & Share:  Send E-mail   Facebook   Twitter   Digg   Yahoo Buzz   Del.icio.us   StumbleUpon   Technorati   Google Buzz   Previous: A letter to Secretary Arne Duncan
Next: Education law 'tales of the weird'

Comments

This is merely the beginning of the testing problems we can expect to see. Once teachers are evaluated on the basis of test scores (the students of Miss Jones in Scarsdale will score at the 99th percentile while Mr. Smith's students in the Bronx score at the 9th) you can be certain they will contest the results in courts of law. And the testing experts will be on their side.

Posted by: Linda/RetiredTeacher | July 19, 2010 3:03 PM | Report abuse

In the debate regarding standardized testing there is rarely any mention to the cost of these tests.

This is a multi billion dollar industry.

When you pay for books for a public school library the books may be used for ten years.

When you pay for standardized testing one year, you will have to pay the same amount next year and the next year. As time goes by you will actually have to pay more each year as prices usually go up and not down.

And this does not even cover the cost of implementing computer systems that will analyze test results and supposedly evaluate teachers. Let us see, student A failed in the 5th grade and now fails in the 6th grade so the effectiveness for the teacher of student A in the 6th grade is _____.

Computer systems are expensive and require software that is expensive. In most case software like this is not a one time charge but includes a yearly software maintenance cost. There are yearly costs for all the staff that will maintain the computers and run the software. You will also need to pay yearly for analysts that can make sense of the data provided by the computer.

Most Americans do not realize that a system to evaluate teachers by test results would have to maintain storage on all of tests taken in the past and use mathematical formulas that have not even been developed to arrive at an evaluation of a teacher based on test scores. And this does not even consider when the systems does not have prior test information on students. Without previous years of test scores these system are not of any value. This means that it may take a few years of collecting test scores to obtain any value at all from the system.

Schools now paying X yearly for standardized tests should consider that the yearly cost of buying systems to evaluate teachers on the basis of test results would probably be 4X.

Americans really have to start understanding the costs and make a decision on whether it is wiser to pay for items that directly effect the quality of teaching or spend large amount of money yearly for standardized testing and computer systems of dubious value.

Posted by: bsallamack | July 19, 2010 3:07 PM | Report abuse

In the debate regarding standardized testing there is rarely any mention to the cost of these tests.

This is a multi billion dollar industry.

When you pay for books for a public school library the books may be used for ten years.

When you pay for standardized testing one year, you will have to pay the same amount next year and the next year. As time goes by you will actually have to pay more each year as prices usually go up and not down.

And this does not even cover the cost of implementing computer systems that will analyze test results and supposedly evaluate teachers. Let us see, student A failed in the 5th grade and now fails in the 6th grade so the effectiveness for the teacher of student A in the 6th grade is _____.

Computer systems are expensive and require software that is expensive. In most case software like this is not a one time charge but includes a yearly software maintenance cost. There are yearly costs for all the staff that will maintain the computers and run the software. You will also need to pay yearly for analysts that can make sense of the data provided by the computer.

Most Americans do not realize that a system to evaluate teachers by test results would have to maintain storage on all of tests taken in the past and use mathematical formulas that have not even been developed to arrive at an evaluation of a teacher based on test scores. And this does not even consider when the systems does not have prior test information on students. Without previous years of test scores these system are not of any value. This means that it may take a few years of collecting test scores to obtain any value at all from the system.

Schools now paying X yearly for standardized tests should consider that the yearly cost of buying systems to evaluate teachers on the basis of test results would probably be 4X.

Americans really have to start understanding the costs and make a decision on whether it is wiser to pay for items that directly effect the quality of teaching or spend large amount of money yearly for standardized testing and computer systems of dubious value.

Posted by: bsallamack | July 19, 2010 3:20 PM | Report abuse

The Federal government should be fully responsible for standardized testing.

States do not need any more Federal government ideas where the Federal government pays only partial costs and states than have to make up the significant difference.

Other nations set up standardized tests and pay the full cost.

The Federal government should set up a system of yearly multiple choice tests in reading and math.

All of the costs should be paid for by the Federal government.

We have already seen the problem of local "standardized" tests in reading and math that have no basis in reality when the results of these tests are compared to the real standardized tests of national tests where claims of 44 percent proficient meet the reality of 14 percent proficient on national tests.

Only the Federal government has the size and resources to keep the costs of standardized testing reasonable.

Any new standardized testing should be discouraged. Costs of standardized testing of science are better spent on better teachings and equipment for science labs. There is already a shortage on qualified science teachers and yet the Federal government wants to money spent on "standardized" tests. Spend the money to deal with the shortage, when everyone knows more qualified science teachers are needed.

There seems very little point in the expensive cost writing tests when this money could be spent to develop programs to help students to learn how to write. A laptop or computer provided to every student would be more useful in learning to write and less expensive than the yearly costs of a "standardized" test on writing.

In fact there appears to be total absurdity in the idea of standardized testing as proposed now by the Federal government.

Current national tests given every two years of reading shows serious problems of Title 1 poverty public schools with failure rates of 56 percent in 4th grade reading.

The Federal government proposes more standardized test in science, history, and writing.

Given the reading failure rates it seems inconceivable that the Federal government is pushing for states to do standardized testing in multiple areas such as science and history. How in the world will tests that require the reading of questions improve the education of student in Title 1 poverty public schools that can not read? Does the Federal government really believe that 49 percent of students in Title 1 poverty public schools that fail the test for 8th grade reading are going to do any better on a 8th grade science test.

We need sense in the national education policy of the Federal government and not absurdity and the prospects of ever spiraling high costs.

Posted by: bsallamack | July 19, 2010 4:11 PM | Report abuse

If it walks and quacks like a scam, it is a scam. Why is it that Pearson is the go-to company for assessments? Tax payers have the right to know.

Posted by: Nikki1231 | July 19, 2010 6:58 PM | Report abuse

I just finished reading the book Making the Grades: My Misadventures in the Standardized Testing Industry, written by Todd Farley (Polipoint Press). It is a must-read for teachers, administrators, school boards and policy wonks. Not only is it scathing and hilarious, it is alarming and horrifying. Having read it, Valerie's descriptions of the testing mess come as no surprise.

Posted by: Incidentally | July 19, 2010 7:01 PM | Report abuse

Pearson is involved with the Maryland School Assessments (MSA).
Is anything happening there?

Posted by: edlharris | July 19, 2010 8:39 PM | Report abuse

"In the debate regarding standardized testing there is rarely any mention to the cost of these tests.

This is a multi billion dollar industry."

Gee, I wonder why Arne Duncan and Michelle Rhee never mention the cost of standardized testing? Could it be that they have connections to the standardized testing companies?

Posted by: jlp19 | July 20, 2010 11:57 AM | Report abuse

just be glad you haven't seen the source code to some of pearsons software products. it's a mess!

Posted by: AuthMan | July 20, 2010 9:42 PM | Report abuse

The comments to this entry are closed.

 
 
RSS Feed
Subscribe to The Post

© 2010 The Washington Post Company