Company's Errors on SAT Scores Raise New Qualms About Testing

[lawyerlee]

This is crazy!

Company's Errors on SAT Scores Raise New Qualms About Testing (http://www.nytimes.com/2006/03/10/education/10sat.html?_r=1&th&emc=th&oref=slogin)
New York Times

By KAREN W. ARENSON
and DIANA B. HENRIQUES
Published: March 10, 2006

The scoring errors disclosed this week on thousands of the College Board's SAT tests were made by a company that is one of the largest players in the exploding standardized testing business, handling millions of tests each year.

The mistakes, which the company, Pearson Educational Measurement, acknowledged yesterday, raised fresh questions about the reliability of the kinds of high-stakes tests that increasingly dominate education at all levels. Neither Pearson, which handles state testing across the country, nor the College Board detected the scoring problems until two students came forward with complaints.

"The story here is not that they made a mistake in the scanning and scoring but that they seem to have no fail-safe to alert them directly and immediately of a mistake," said Marilee Jones, dean of admissions at the Massachusetts Institute of Technology. "To depend on test-takers who challenge the scores to learn about system failure is not good."

These were not the first major scoring problems that Pearson has experienced. The company agreed in 2002 to settle a large lawsuit over errors in scoring 8,000 tests in Minnesota that prevented several hundred high school seniors from graduating. It also has made significant scoring errors in Washington and Virginia.

It's sad to think that the only reason they found out about the mistakes was because students challenged their scores. I would never have felt confident enough to challenge my scores on any standardized test I've taken. And I probably would have made fun of people who did because it just seems so out there. But I guess it was a good thing that these people questioned their results.

[kam]

I am vehemently opposed to standardized tests as the deciding factor unless you are applying for a technology/math position somewhere or to a tech school. I heard on the news that several colleges are now having to re-review their denied applicants and their applicants who got into the school but didn't get financial aid as a result of this.

[Asha]

as i have said before in another threads, standardized achievement tests have shown weak correlations with any type of instruction but strong correlations with a student's socioeconomic status.

also, my professor was a expert witness about sat's showing test bias for women and minority groups and there is strong statistical evidence that supports that claim.

also, there are many psychometric methods to measure reliability, and i am not sure which meausure the sat uses. some meausures of reliability are stronger than others.

[lawyerlee]

1,600 More SAT Scoring Problems Found (http://www.washingtonpost.com/wp-dyn/content/article/2006/03/14/AR2006031400608_pf.html)
Washington Post

By JUSTIN POPE
The Associated Press
Tuesday, March 14, 2006; 11:07 AM

The College Board disclosed Tuesday that an additional 1,600 SAT scores have not been rechecked from an exam in October that had scoring problems.

The previously overlooked batch of answer sheets came from among those being scored separately for a variety of reasons, including security concerns. Some of those scores were on hold and had not been reported, but others may have been reported incorrectly, according to an e-mail sent to college admissions officers and guidance counselors early Tuesday. The statement also was posted on the College Board's Web site.

College Board spokeswoman Chiara Coletti said Tuesday she became aware of the latest problem late Monday. The sheets will be rescanned over the next few days and colleges and students notified of any changes as soon as possible.

She said she expected some, though perhaps a small number, of scores would be affected.

[kam]

Wha wha wha???????

If I were a student who took these tests, I'd be challenging them even if I thought they were right at this point.

[jnettie]

Being only 5 points shy is enough to screw you out of a scholarship or entrance into a school. I know that the college I work for is extremely strict on this. This is total crap.

[lawyerlee]

More news on this today:

SAT Problems Even Larger Than Reported (http://www.nytimes.com/2006/03/23/education/23sat.html?th=&emc=th&pagewanted=print)
New York Times
By KAREN W. ARENSON

The College Board disclosed yesterday that the problems resulting from the misscoring of its October SAT examination were larger than it had previously reported.

In a statement, the organization said it discovered last weekend that 27,000 of the 495,000 October tests had not been rechecked for errors. It said that after checking those exams and one other overlooked set, it had found that 400 more students than previously reported had received scores that were too low.

A board official added that the maximum error was 450 points, not 400.

This is the third time in two weeks that the board, which administers the exam, has acknowledged that its earlier assessment of the problems was wrong. In its statement, the board also outlined steps it planned to avoid mistakes.

The disclosures prompted fresh criticism that the board had not been as forthcoming as it should have been in disclosing the problems promptly and in detail.

"Everybody appears to be telling half-truths, and that erodes confidence in the College Board," said Bruce J. Poch, vice president and dean of admissions at Pomona College in Claremont, Calif. "It looks like they hired the people who used to do the books for Enron. My next question is what other surprise we're going to hear about next."

[dionysia]

I am very proud that my alma mater made SAT scores optional for admissions.

(Mount Holyoke College)

Di

[lawyerlee]

I am very proud that my alma mater made SAT scores optional for admissions.

(Mount Holyoke College)

Di
That's cool. :) I wonder how many other schools are considering this change in light of not only these mistakes, but what seems to have been a series of attempts to cover up the extent of them and/or an inability to get a handle on the problems. I hope many schools will see just how much can go wrong.

[dionysia]

Here's an article that the MHC President wrote on the subject:
http://www.mtholyoke.edu/offices/comm/oped/creightonla.shtml

Di

[fuzzy]

I wonder how many other schools are considering this change in light of not only these mistakes, but what seems to have been a series of attempts to cover up the extent of them and/or an inability to get a handle on the problems.

A lot are considering, although I don't know how many will follow through with making their institutions SAT-optional. I know here (here being where I work), we went SAT-option five years ago and have experienced in increase in the quality of applicants since. I'm told other STA-optional schools experienced the same shift when they introduced their optional policies, which I think is interesting.

[SingleWhiteFemale]

I find it interesting that so many of the news outlets made it seem that the test was *the* factor to getting into college. As if there was some magical cut off number, and nothing else mattered in the person's application. It drives me apesh!t that some kids are brainwashed into thinking this score is the only thing that matters into getting into a school--that the 4 years of high school grades, activites, volunteer work, and essays are nothing. While there are errors with the SAT and scoring, I see a bigger flaw in the way this stupid test is being marketed. That it is the end all be all and only factor that matters. The College Board makes loads of money from such a mentality (go to the bookstore right before the SAT/AP tests and the entire section is sold out), so I guess they don't want it to change.

I do know of the boy with a 1550 with a 2.5 GPA and 3 musicals as his extracurriculars, that got into a few, non-selective schools (Yale, Harvard, UNC, Michigan, Stanford, and a few of the UCali school rejected him). And the girl with the 4.5 GPA and 1370, a few volunteer groups, a few activities she was dedicated to that got into Harvard, Princeton, Hopkins, UMaryland. But you have the 3.8 GPA, 1270 with a sport and 1 club that got into Notre Dame, Columbia, UMaryland, UNC-CH, and Cornell. Or the 3.4 GPA with a 1100 who sat on their duff in HS that got into Cornell with a full-tuition ride (parents are college grads, not alumni of Cornell). It is a true crapshoot.

It also seems that as the colleges become more selective, students are getting rejected and looking for an external force to blame. That they're the sh!t and the school messed up not accepting them--that it couldn't possibly be some shortcoming of their own as the reason they're not in.

Can ya tell I hate the admissions proccess? :-D

[lawyerlee]

It also seems that as the colleges become more selective, students are getting rejected and looking for an external force to blame. That they're the sh!t and the school messed up not accepting them--that it couldn't possibly be some shortcoming of their own as the reason they're not in.
I don't think it is blaming other people if there were in fact grading problems associated with the test and your score on that test is a factor in whether you are granted admission. I'm, of course, not arguing that it is necessary or desirable to give these test scores so much weight. But it is important that the scores are accurate if they are going to be considered. There is a whole other question of the cultural relativism inherent in these tests that we aren't even discussing here that *really* puts me off them and relying on them so heavily.