Jonah Lehrer, in April of 2011 -
We live in a society obsessed with maximum performance. Think of exams like the SAT and the GRE. Though these tests take only a few hours, they’re supposed to give schools and companies a snapshot of an individual’s abiding talents.
[…]
But as Mr. Sackett demonstrated with those supermarket cashiers, such high-stakes tests are often spectacularly bad at predicting performance in the real world. Though the SAT does a decent job of predicting the grades of college freshmen—the test accounts for about 12% of the individual variation in grade point average—it is much less effective at predicting levels of achievement after graduation.
[…]
The reason maximal measures are such bad predictors is rooted in what these tests don’t measure. It turns out that many of the most important factors for life success are character traits, such as grit and self-control, and these can’t be measured quickly.
Jonah Lehrer, in June of 2012 -
This trend held for many of the specific biases, indicating that smarter people (at least as measured by S.A.T. scores) and those more likely to engage in deliberation were slightly more vulnerable to common mental mistakes.
If SAT scores aren’t that great at what they’re supposed to do - namely, predicting achievement. And if it’s a bad idea to measure for maximal performance, rather than typical performance, doesn’t it follow that SAT scores are a lousy proxy for intelligence?