Cheating on the SAT

The thing I can’t get out of my head when I read this NYT article on kids paying other kids to take the SAT for them is simply this:

All they charged was $3600? Cripes, that’s a bargain at ten times the price.

Don’t get me wrong: you shouldn’t cheat, both because it’s wrong (which should be enough reason) and tactically too risky (in case the first argument wasn’t enough).

But I mean, come on, let’s do the math: a one-percent increase in salary over your life is easily a five-digit number even if you’re kind of a slacker. Two significantly different SAT scores mean admission to schools of two significantly different calibers. And I doubt the salary increase we’re talking about here is just 1%.

For those of you who are fans of the Drake equation, which uses best-guesses to try to figure out whether there’s intelligent life out there, I challenge you to apply this reasoning to SAT prep.

In fact, you might even try to create an analogous equation governing this stuff, like I just did. (I hope you have more luck than I did; if so, please let me know.) But, equations aside, it’s not really that hard to think about.

To figure out what a higher SAT score is worth, just do the following steps:

First, get a lifetime earnings calculator. (Google it; there are many.)

Then, use it to estimate the student’s lifetime earnings, given that he or she attends the best school to which he or she can gain admission given the initial SAT scores.

Then, take the average (expected) gain in SAT scores given a particular preparation method.

Then, use the calculator to estimate the student’s lifetime earnings, given that he or she attends the best school to which he or she can gain admission given the final (expected) SAT scores.

The difference between the two lifetime earnings is the value of the higher SAT score.

And now that I’ve said all that out loud, I’m starting to realize that “only four-digit” prices for SAT prep only make sense for providers who can offer only single-percentage-point gains with a high variance, as delineated in this article in the Wall Street Journal regarding the average benefit of SAT prep.

Now these days, I am no longer doing SAT prep, having left Bodsat Prep in 2016. However, the work I do preparing students for the AMC competitions (as well as the Math 2 and Physics SAT Subject Tests) still seems to be governed by this math.

Since we don’t often see prices like this, the conclusion I come to is: almost no one is delivering reliable results. (Or the people who are aren’t also good at pricing.) Interesting.

The power of the nudge

My friend and colleague Jai Flicker just posted this lovely essay on the time he knew just what to say to a student to make a big difference. His point was that one’s attitude towards work matters a lot. I take away a different point: that there’s more to teaching than knowing the material and knowing how to teach.

Don’t get me wrong. I think that knowing the material and knowing how to teach are both higher bars than we give them credit for.  (“Knowing how to teach” is especially tricky, it turns out.) But if teaching is your career, then I think that those two important skills, while necessary, aren’t sufficient on their own.

Most teachers never studied Buddhist philosophy, Qigong, or Non-Violent Communication, but Jai sure did. I think it’s clear from even a casual interaction with Jai that part of what makes him an exceptional educator — his “special sauce” if you will — is his awareness of the effect of emotional state and perspective on outcomes.

For me, it might be focus on effective processes of creative thought (because of my experience in music performance and competitive puzzle-solving), or maybe my love of analogy.  Or my great fashion sense* — it’s hard to know these things about yourself.  (Indeed, I just asked a few trusted colleagues this question, and learned rather a lot.)

I sure know what it is in others, though.  For each of my students and for each of my professional colleagues, I know what makes him or her special — what special sauce each brings to a hurdle, from home schooling to playing first violin; from fearless honesty to boundless patience; from experience as a youth counselor to experience as a corporate executive.

It’s important because of the power of the nudge.  Any good educator can help you with the next step, but the right idea, at the right time, presented in the right way: this is what helps you make a huge leap instead of another small step.  Great educators try for the leap instead of relying solely on the step.

But how is that done? I think it takes three components, two obvious, and one not: being a good teacher and deep subject knowledge are the table stakes. But it’s outside experience that makes the difference.

Jai has his mindfulness practice; I have competitive puzzle solving; others have other sources of “special sauce.” That’s always been an important part of my teaching practice: “I teach people, not just math” is how I’ve tried to explain it.

And increasingly, that’s the basis for my referrals too: I’m looking to catalyze leaps, and that means knowing my network not just as professionals, but as people.  When they go to work, they bring their whole selves, and that’s why they get such better results.

*PS I actually have terrible fashion sense.

– – –

Here’s my favorite one-session story.

Rachel and David came to me because Rachel, a straight-A student for years, just could not crack a B in her precalculus course. Nothing she did was enough; she kept getting docked too many points on tests; her motivation was faltering and so was her meticulousness.

It was apparent that Rachel was a mature and grounded student, and that David was an involved and supportive but not at all “helicoptery” parent.

We talked about the class itself, and the teacher in specific.  It came out that the teacher came of age in Soviet Russia towards the end of the Cold War. I know a bit about that culture, because it was a particularly noteworthy time in the history of math competitions, which I coach.

So Rachel and I talked a bit about that period, about the relentless focus on grit, drillwork, and obedience, about the aspects of those qualities that we value in our culture today, and about what a teacher who was a student during that period might bring to her teaching practice — what expectations she might have, and how she might gauge whether students are measuring up as they should.

We looked at one of Rachel’s recent tests through that lens, and instead of undeserved points lost due to unimportant minor mistakes, we both saw warning signs of a student in command of only part of the end-to-end process that leads to reliably correct answers.

About two weeks after that session, I called to find out why they hadn’t booked a second session, and the answer was a delight: it was because Rachel had only gotten an “A” on every homework and quiz since our meeting.

(I later helped her improve her SAT scores, but that’s another story.)

What the SAT tests

What the SAT tests

I can’t make this stuff up: three days after I reposted my 2012 essay in response to an article by a grown-up taking the SAT, Forbes published this article about — you guessed it — a (different) grown-up taking the (new) SAT. I can’t resist the karmically-ordained opportunity to comment.

First, let me address the assertion “it’s all about speed”: no controversy there; it’s completely correct. Though I would frame it as “the SAT tests fluency over surface knowledge,” it’s the same thing.  You can’t spend 30 seconds (i.e. an eternity!) trying to remember a formula on this test.

On a related point, let me just observe that this is a trait that the “revised SAT” — first given in March 2016 — shares with its primary competitor the ACT, but not with its predecessor (at least, not to this degree).

Next up: “The test is designed to trip you up if you work too quickly (and make) a careless error of a particular type.”
Also true.  The SAT is a test of meticulousness, and has been for over a decade.  Perhaps even for its whole history.

“Doing well on the SAT requires that you know the tricks of the test, and that you’ve memorized many formulas so that they come to mind instantly.”
Yes, but those are hardly “tricks of the test.” Knowing the formulas that they tell you to know is table stakes, nothing more.

“…they haven’t yet figured out how to construct good stats questions.”
That may be true, but the author’s example rings hollow: sampling only 117 people from a “large town” is simply not the same order of error as surveying a local restaurant on a Saturday to determine the population’s sports-watching preferences. After all, that restaurant either has the game on or it doesn’t; either way, there’s a clear sampling error, the cardinal sin of statistics.

“…there’s not a single question on any of the practice math SAT exams that I would call difficult.”
I’m probably supposed to debate this point, but there’s no sense in doing so, because he’s basically spot-on. At the core, this is the chief problem with the test, at least for top-tier students. Always has been, and it’s worse now than before. The test is simply not good at distinguishing very good students from great thinkers, nor very good thinkers from great thinkers.

But what college admissions needs more than to make the SAT optional (as the author advocates) is to replace it or at least supplement it with better tests of high achievement.  As the author himself points out, “they could go a long way towards a better test by simply giving students twice as much time. If that made the test too long, they could simply ask fewer questions.”

Right you are, sir. And it’s been done! That’s the very reason for which tests like the AMC12, USAMTS, and AIME (and to a lesser degree the Math 2 Subject Test) exist: to give truly outstanding students an opportunity to shine based on math ability rather than speed and meticulousness under pressure.


Take it like a grown-up

A few years ago, a 35-year-old took the SAT and wrote about the experience.

I responded with the essay below, which I think is relevant to most academic work, not just the SAT.

I posted the essay on Facebook, where a very interesting discussion ensued.

Forgive me if this offends:

Being scared of the SAT is an uninformed response to the serious and real issue of cultural test-mania (unless you are getting paid to write a funny article in which you aim to be uninformed, as in this case). The way you deal with the SAT is the same way you deal with any of the other problems you deal with as an adult: you learn about the most important ideas from a reputable source, you do the minimum necessary preparation, you make a backup plan in case you need to take it more than once, then you execute the plan.

I mean, seriously, taking the SAT cold is as smart as doing your taxes cold. For your taxes, you buy software, or you hire an expert, or you google some basic questions. Same applies here: the basic math facts you need fit onto a page. The reason for the 0.25 point penalty (and the way to use it to your advantage) takes three paragraphs to explain, tops. The fact that the essay comes first is well-understood to be a mind game (and again, one you can use to your advantage).

A little research, folks. That’s all it takes. The big bucks in private prep are spent by three categories of people: people who actually need every last point, people who need to save every possible minute, and people too lazy or afraid to search the web for basic information.

It’s honestly not rocket science. It’s wholly doable with some forethought. Treat it as such instead of freaking out. In other words, take it like a grown-up.