ACT & SAT: are they winners’ games?

I didn’t expect Farnam Street to relate so directly to college admissions, but it does, and in a way that is as important as it is deep.

According to the post linked above, the outcome of a “winner’s game” is determined mostly by the actions of the winner; the outcome of a “loser’s game” is determined mostly by the actions of the loser.

So, in a winner’s game, superior skill matters, whereas in a loser’s game, it’s consistency that matters.

Everyone acts as though the SAT and ACT are winners’ games.  And, for middle-tier scorers, they probably are.  But for high scorers, these tests are loser’s games, 100%.

In other words, you don’t get from 1400 to 1550 by becoming smarter or more knowledgeable. You get there by consistently avoiding mistakes.

You show me a 1400 scorer, and I’ll show you someone who already has the necessary knowledge, but who also experiences some combination of anxiety, hubris, wandering attention, and/or magical thinking when it comes to answering the hard questions.

Getting rid of such bad habits is really hard. That’s why so few people get perfect scores.

That’s also why high scores are so prized.

Cheating on the SAT

The thing I can’t get out of my head when I read this NYT article on kids paying other kids to take the SAT for them is simply this:

All they charged was $3600? Cripes, that’s a bargain at ten times the price.

Don’t get me wrong: you shouldn’t cheat, both because it’s wrong (which should be enough reason) and tactically too risky (in case the first argument wasn’t enough).

But I mean, come on, let’s do the math: a one-percent increase in salary over your life is easily a five-digit number even if you’re kind of a slacker. Two significantly different SAT scores mean admission to schools of two significantly different calibers. And I doubt the salary increase we’re talking about here is just 1%.

For those of you who are fans of the Drake equation, which uses best-guesses to try to figure out whether there’s intelligent life out there, I challenge you to apply this reasoning to SAT prep.

In fact, you might even try to create an analogous equation governing this stuff, like I just did. (I hope you have more luck than I did; if so, please let me know.) But, equations aside, it’s not really that hard to think about.

To figure out what a higher SAT score is worth, just do the following steps:

First, get a lifetime earnings calculator. (Google it; there are many.)

Then, use it to estimate the student’s lifetime earnings, given that he or she attends the best school to which he or she can gain admission given the initial SAT scores.

Then, take the average (expected) gain in SAT scores given a particular preparation method.

Then, use the calculator to estimate the student’s lifetime earnings, given that he or she attends the best school to which he or she can gain admission given the final (expected) SAT scores.

The difference between the two lifetime earnings is the value of the higher SAT score.

And now that I’ve said all that out loud, I’m starting to realize that “only four-digit” prices for SAT prep only make sense for providers who can offer only single-percentage-point gains with a high variance, as delineated in this article in the Wall Street Journal regarding the average benefit of SAT prep.

Now these days, I am no longer doing SAT prep, having left Bodsat Prep in 2016. However, the work I do preparing students for the AMC competitions (as well as the Math 2 and Physics SAT Subject Tests) still seems to be governed by this math.

Since we don’t often see prices like this, the conclusion I come to is: almost no one is delivering reliable results. (Or the people who are aren’t also good at pricing.) Interesting.