Link to jump to start of content The Seattle Times Company Jobs Autos Homes Rentals NWsource Classifieds
The Seattle Times Education
Traffic | Weather | Your account Movies | Restaurants | Today's events

Sunday, March 12, 2006 - Page updated at 12:00 AM


SAT error highlights imperfections — and raises questions of risk tolerance

The Associated Press

For the past five years, Hamilton College in upstate New York has been one of a growing number of colleges not to require the SAT exam. The test causes too much anxiety, Hamilton concluded, and there's a risk of missing bright students who don't test well.

On Tuesday night, Hamilton's faculty voted unanimously to make that policy permanent. By coincidence, the next morning brought a reminder that there's another potential downside to standardized tests: News arrived that 4,000 SAT exams taken last October had been graded incorrectly.

"They do a lot of things right," Hamilton dean of admission and financial aid Monica Inzer said of the College Board, which owns the exam. "But it shows how vulnerable we all are when we depend too much on one test."

The error affected less than 1 percent of test-takers and isn't expected to affect admissions decisions, though Inzer noted it's too late for students to apply to schools they might have considered with a higher score.

Experts say mistakes are inevitable in any operation on the scale of grading millions of tests. Still, the episode is likely to spark wider discussion about standardized tests, college-entrance exams and the growing number of high-stakes, state-level exams: Just how much risk of error is tolerable when students' futures are at stake?

Recent years have seen a number of scoring errors on state-level tests and graduate-school exams such as the Graduate Management Admission Test. Some were small and caught early, others significant.

In 2003 and 2004, 4,100 people were incorrectly told by the Educational Testing Service that they had failed a teacher-licensing exam. In 2000, more than 8,000 Minnesota high-school students were mistakenly told that they had failed a state exam, and dozens missed their class graduation ceremonies.

That mistake prompted a previous incarnation of Pearson Educational Management, which also scores the SAT, to pay a $7 million settlement. On college-admissions bulletin boards last week, there was talk of lawsuits in response to the SAT mistake, along with angry comments from students and parents.

While the SAT error was comparatively small, "it is such a visible program that people freak out," said Scott Marion, vice president for the New Hampshire-based National Center for the Improvement of Educational Assessment.

Critics of standardized testing seized on the error as confirmation that the testing industry — dominated by CTB/McGraw-Hill, Harcourt Assessment and Pearson — is stretched too thin for the public's good.

"The volume is way up, and the people with the competence to do this don't exist," said Robert Schaeffer of the group Fair Test, which opposes many of the ways standardized tests are used.

A recent report by Education Sector, a Washington, D.C.-based think tank, portrayed a highly competitive industry facing huge pressure from its biggest clients — the states — to cut costs and deliver results quickly. That pressure is sometimes reinforced by contract provision for financial penalties if scores are late.

Pearson says the SAT error may have been caused by excessive moisture that caused answer sheets to expand and some marks to be unreadable. Spokesman David Hakensen said Friday that Pearson has invested heavily in quality and capacity; since 2000, it has increased its number of scanners by 66 percent, added 60 percent more processing space and increased its report-printing capacity 45 percent.

"We take any mistake seriously and we feel terrible about it, of course," he said. "The people administering this test are people too and are aware that this is important stuff and feel bad when this happens."

Most of the incorrect scores were off by fewer than 100 points on the 2,400-point test, and only 16 changed by 200 points or more, the College Board said.

Marion said companies such as Pearson are improving their processes, but the increased demand and time pressure may be negating the progress. In any case, perfection is impossible. "You won't see this mistake from Pearson again, but you'll see a different mistake," he said. "As long as you have humans involved, you're going to have some mistakes."

Fair Test, Schaeffer's group, wants more transparency and expansion of the rights of students to challenge their scores on standardized tests.

The SAT error was uncovered because at least one student asked for a hand score. But that request costs $50 (refunded if an error is found), and there's a risk of getting a lower score. The College Board says it gets about 500 such requests a year, most of which reveal no error.

Copyright © 2006 The Seattle Times Company




More shopping