A few thoughts about the ACT/SAT

by | Sep 22, 2021 | Blog, college admissions, SAT prep, test prep

If you’re heading to college, you’re almost certainly going to take the ACT or SAT at least once. I’d like to set the record straight on a few things so that we can all minimize unproductive time and effort, especially in light of the fact that most students take these tests during school years that are already very busy.

On preparation

ACT/SAT preparation is a big industry encompassing a wide range of options. I don’t think that “low quality” providers are terribly common, but I do think that it’s important to match a provider to your needs, because there is a wide range, and fit matters. There are three issues in particular that come up around ACT/SAT prep that I think are worth calling out:

  1. On variable depth: Lots of ACT/SAT prep is done by those who did well on the tests themselves but whose teaching experience (and, sometimes, incentive to ensure student success) can vary widely. If you are unaware, this can cause problems. If you are savvy about it, though, this can be an opportunity to get what you need for less money overall. 
  2. On score bands: What you need to improve your score is not only affected by what kind of person you are. It’s also affected by where you are on the score curve. For example, tell me where you are scoring, and I’ll tell you whether vocabulary, grammar rules, reading style, or attention management are probably costing you the most points on the verbal sections of these tests. Some teachers are aware of this and some aren’t; some specialize in a narrow score band and some don’t. But whenever I talk with a parent for the first time, I note whether they have considered this important part of the decision process. 
  3. On the grapevine: It’s great to be well-connected! Just be aware that Suzy’s great tutor may or may not have actually been a great tutor. Maybe she was just great for Suzie. (This is also why I encourage tutors to be selective about whom they take on. “Taking all comers” is good in the beginning, but it isn’t a good long-term plan for most.)

Basic pro facts

Here is some “inside baseball” about these tests. Most of the pro tutors I know are delighted when clients have already considered these ideas:

  • No school cares which test you pick. Really. (If you have a reputable college counselor telling you otherwise, then please trust that you’re in a very specific and unusual situation.) So just pick the one you do best on (see below), or pick the one whose test date happens when you need it to, and move on. 
  • There is no “penalty for guessing.” Neither test currently assigns negative fractional points for incorrect answers. Which is unfortunate, because you’d be better off if they did. This video helps to explain why that’s true (even though it may seem “obviously” false). The punch line is: without “penalty” points, you’re forced to spend the last minute of your testing time bubbling in random answers to whichever questions you haven’t yet answered. 
  • Most students will get comparable scores on the two tests. Of those who do better on one or the other, it’s a 50/50 chance which one you’d do better on (see also “subjective preference” below). The best way to determine whether you are a student who does better on one or the other is to take one of each in comparable circumstances and compare the two scores using a concordance table from either organization. 

That said, if you believe that taking both tests is not in your best interests (e.g. because you have very limited time), then here are some rules of thumb:

  • If you read especially slowly (e.g. have a tested learning difference, esp. low processing speed) then the ACT is going to penalize you extra hard for it. Prefer the SAT.
  • If you are considered a linear thinker and/or your primary school strategy is to “grind it out,” then the ACT is going to be more in your wheelhouse. The ACT lends itself better to simply learning all the material and techniques, then delivering on test day. Prefer the ACT. 
  • Conversely, if you are considered a good test-taker and/or your primary school strategy is to wing it, this suggests a preference for the SAT, which remains slightly more gameable in this way. Prefer the SAT.
  • Crucially, most students express a subjective preference for the ACT. This is not to be confused with “most students do better on the ACT,” which is simply not true; scores are normed to reflect percentile rankings. 

This last point might be worth expanding a bit. 

In other words, schools aren’t looking at your raw scores per se; they’re looking to see whether you did better than most others. The all-too-popular canard that “the ACT is better for most students” is not only wrong, it betrays a deep misunderstanding of the whole situation. The score is normed across all test-takers. That preference, therefore, gets normed out of the score. In other words, the ACT “feels better” to everyone else too, and so thanks to the norming that happens across this particular zero-sum game, it’ll all come out in the wash. 

And let me emphasize: the best way to select is to take one of each test in comparable circumstances, and then to compare the two scores using a concordance table from either organization. 

As I say to students who are ready to hear it: the hard truth here is that you’re not here to have a good time; you’re here to get a job done, and that means a good score. So don’t let your preferences guide your decision unless they are likely to affect your outcome. Your score is the thing to focus on. Structure your preparation (and your test selection) in the way that will lead to the outcome you want. 

This framing helps put a more human face on the underlying mathematical truth about the way these tests are scored, and on the need in life to be able to jump through the occasional tricky hoop.

Some thoughts for top students

If you’re in the top tenth or so of students (i.e. if you test at the 90th percentile or higher), then the game is a bit different for you, since it’s no longer about being able to solve problems that few others can. Instead, for you, it’s about not making avoidable errors.

In other words, as you move to the top of the curves, these tests go from being winners’ games to being losers’ games. 

Let me unpack that a bit. The people who score in the middle of the pack — say, an SAT 1200 or an ACT 21 — are playing winners’ games, where doing better relies on getting more problems right. By contrast, the people who score at the top of the pack — say 1400+ on the SAT or 28+ on the ACT — are playing losers’ games, where doing better relies on just performing consistently while others make mistakes.

This means that beyond a certain point, the winning strategy shifts from trying to get more right, to trying to make fewer unforced errors.

This is where preparation passes outside the realm of what most of us think of as fact- and tactics-based “test prep” and into softer skills adjacent to psychology, habit formation, and self-awareness.

I created a short (ten minutes maybe?) free mini-course on this called What Am I Missing? that you are welcome to give to anyone you’d like, and to take yourself. It gets into more detail than comfortably fits here.

Consider the long game

Parents, please take an uncomfortable moment to consider the unlikely scenario where the student wins admission to a school that’s hard enough that they wind up dropping out. Now consider the less dramatic but much more common version: the school is hard enough that they wind up needing continued help throughout their undergraduate years and even beyond. (These do happen; I’ve seen both first-hand.)

My point is that we’re all working together here to execute a successful launch. This means both success and actual launch.

One of the ways this can positively affect SAT/ACT prep for active parents and their kids is the explicit recognition that a bad run at these tests can in many cases be a very positive milestone in the long run. There’s nothing like an unexpectedly difficult run at an important test to challenge expectations and catalyze a bit more self-awareness. After all, if the SAT/ACT is hard enough to shake you up, what does that say about every test you are going to face over the next four years?

Now is the time to either downshift or step up. It is absolutely possible to gain new knowledge, to gain new skills, to try on new perspectives, to embody new strategies, to experiment in the service of self-improvement. 

You’re about to do that for four years. Why not start right now? 

But again, you have to see this as a long game in order to get these long-term benefits from these immediate disappointments.

A historical side-note

Originally, the SAT was an aptitude test and the ACT was an achievement test. 

  • The SAT has its origins in trying to assess which enlisted soldiers were officer material (and which didn’t belong in the army in the first place). I.e. assessing undemonstrated potential. 
  • The ACT, by contrast, has always aimed to assess what material a student actually knows, as a fraction of the material that the student is supposed to know. 

In other words, the SAT tried to determine how capable you were in general, and the ACT tried to determine whether you’d already learned what you were supposed to have learned by then. The tests have evolved over time (mostly with the SAT drifting towards the ACT), but there are still some shadows of their origins in their modern incarnations. 

Radiolab did a great series on this and a few related issues. But please note: it sometimes gets almost as dark as it is consistently fascinating.

Other quick links

  • I do SAT and ACT prep professionally, especially for high scorers. My buddy Ted Dorsey (of Wes & Ted’s Excellent Conversation) does even more, and across a wider range of student types. My former company bodsat.com has some very good programs that I helped design. You can call any of us, and if we aren’t the right fit for you, we’ll steer you in the right direction.
  • I know a lot of specialists who can help in various ways, and I see connecting people as an important (and enjoyable!) duty. If you need specific help, let me know what you’re looking for and I’ll do my best. And if you’re a fellow professional, please get in touch to let me know what you do best, so that I can help connect you with families who need it.
  • I mentioned my quick, free mini-course “What Am I Missing?” earlier. I’m linking it here as well: wescarroll.com/waim. Feel free to explore it, and to share it.

Quote of the week, from an adult student:

”I missed this one because I (stupidly) assumed that…”

Don’t be so hard on yourself, sister. Assumptions are just shortcuts that didn’t ace their job. I would say that you jumped to a conclusion, didn’t notice that you’d jumped, and so you didn’t think to question it. Happens to the best of us. Let me help you up, dust yourself off, and let’s try again, hey? Right on.

Improvement means being wrong a lot. That’s not stupidity. It’s just the “cost of doing business” when you’re trying to get better.

The key bit is that you examine why that unhelpful assumption was poised for you to grab onto in the first place. It was probably there for (what used to be) a good reason. Re-assessing the reasoning, and possibly re-visiting your conclusion, is a very helpful way to keep no-longer-helpful habits and assumptions from continuing to re-assert themselves.

Last words

Hey, thanks for showing up here. I hope you found it helpful, or at least interesting.

I’m [email protected], I read all feedback personally, and I really do care whether this was helpful, and how I can make it better next time around.

Let’s talk about the AMC.

by | Aug 25, 2021 | Blog, college admissions, SAT prep, test prep

They say that the best time to plant a tree is 20 years ago, but today would be second-best. That’s how I feel about writing about the American Math Competition (AMC): I wish I had spent more time writing about it in the past, but I’ll do the next best thing and write right now.

Who cares?

You do, if you have (or are, or know) a student in high school who is interested in STEM and generally good at school. 

That student (plus their support structure) is usually focused on these three goals:

  1. Get good at core skills needed through adult life
  2. Build out a resume that opens doors later
  3. Get through this craziness without excessive friction or stress

For these students, the AMC remains a little-known but very powerful tool for helping to accomplish these goals.

Can this wait?

Not if you want to participate this school year. The AMC 10 and AMC 12 exams, usually given in February, are being given in November this year. This means much less time to prepare, but crucially, it also means that participating seniors will get to use their good scores in the college admissions process. 

Another benefit: those who participate this year will have much more time to prepare for the second level of the competition (the American Invitational Math Exam, or AIME) in March, if they qualify in November. In past years, students would discover they qualified for AIME with a week or less to spare before competing. This year, competitors will have months to prepare.

This matters for a couple reasons, but the one most focus on is that top schools want to know when an applicant made the cut for AIME. It’s one of those distinguishers that can really matter, esp. for schools with STEM-focused reputations and/or “elite standing.”

Recent testing trends

  • Over the last 20 years, high school has generally become a more systematized “accomplishment treadmill,” with the advantages and disadvantages that this implies
  • Over the last several years, the ACT and SAT have taken significant credibility hits. These include equity issues, doubts about correlation to college success, rejection by major universities, and most recently logistical difficulties simply being able to sign up for and take the test 
  • Side note: one problem with these tests that has always been lurking under the surface is that whatever it is that they test, they test it best for the people in the middle of its bell curve, but their results are most scrutinized for the people at the top of its bell curve. I.e. a harder test would be better-suited for the people who need it most
  • Nonetheless, competition for college admissions has remained very high 

College admissions offices, understanding all these trends, are continuing to use the AMC as a distinguishing characteristic among otherwise hard-to-distinguish strong students. 

I expect that the AMC will continue to increase in prominence in the coming years.

What I observe

Students who prepare for the ACT or SAT tend prepare by filling knowledge gaps, which is all to the good. For most students, this remains the main focus throughout their preparation.

Students at the high end of the curve who prepare seriously, though, only start their preparation the same way. Before too long, they run out of facts to learn and techniques to leverage, because the test simply doesn’t cover that much material that they haven’t already learned. (This is a big part of what I meant when I said that the test is geared towards students in the middle of its curve, but the students who care about it most are the ones at the top of its curve.)

This is the point at which things start going sideways:

  • Preparation becomes an exercise in meticulousness, sometime to the exclusion of engaged critical thinking. For these students, these tests are about simply not letting their attention drift as they answer easy question after easy question. (In my view, this selects for the wrong skills and habits among “top” students, and sets them up for trouble later. If you see it differently, I recommend the movie Brazil. It’s really good, and I hope you like it as much as I do, even though it makes me cringe to watch it.)
  • Also, preparation often becomes an exercise in anxiety management. I feel better about this consequence, since anxiety management is a real-life skill for sure. But I think we can all agree that that’s not the thing we all want to be selecting for in this context either. 

I’ve also observed that students preparing for tests that really challenge them develop skills that in my view hold much higher value overall: critical thinking, self-awareness and self-management, self-evaluation and metacognition, and creative problem-solving.

What I sometimes advise

If the ACT or SAT isn’t hard enough to allow you to show off your real skills, and you like STEM, then the AMC is probably worth looking at:

  • You get to work on problems that are harder than anything your teacher would subject your whole class to
  • You get to explore interconnections in the math you already know without having to learn a ton of new material
  • You get to practice the critical skill of making progress even on incomplete information 
  • Top universities actively solicit your AMC 12 score. This tells you everything you need to know about its importance to them

Recent trends in the AMC

At the top levels, the AMC can be seen as a feeder program for the International Math Olympiad. The US, Russia, and China have been vying for top honors at the IMO for the last few years now. I genuinely don’t know how much of this should be attributed to good coaching; I’m impressed by head coach Po-Shen Loh — he’s clearly focused on teaching well instead of merely on being smart, and you’d be shocked at how many miss that — but that’s all I know, because I’m not anywhere near the inner circle at that level.

But what I do know is that Richard Rusczyk’s project Art of Problem Solving has become the dominant force in this area. What started as a pair of (excellent) books and a thriving community has become the 900-pound gorilla in the space; practically every serious student competitor has some relationship with AoPS. There’s a ton of good stuff to say about AoPS, but I’ll also mention that two false impressions often come out of students’ initial foray there:

  1. “There’s a ton of math I need to learn in order to be successful here.”
  2. “Reading through other people’s solutions is the best way to learn.”

Regarding #1, it would be more correct to say that AoPS will help you learn a ton of math, and a relatively small slice of it is needed for AMC success. Figuring out what’s important remains more art than science, data analysis notwithstanding, because — at least at the AMC level — doing a good job of using what you know is generally more valuable than knowing more.

And regarding #2, AoPS rightly discourages this perspective, and actively, but you can only do so much when such a large proportion of your site is dedicated to the thing you’re telling everyone to use only judiciously. I mean, it’s been said plenty before: math is not a spectator sport. Doing math — and analyzing your steps and missteps after the fact — is the way to get good at it. Deliberate practice isn’t easy, often isn’t fun, and it certainly isn’t your default behavior when you’re reading a web site. But it’s the best way. Unfortunately, almost no students get this, at least not at the early levels.

A few related videos

I’m happy to help directly

  • There’s no charge for a scheduled video chat with me. (If you have a personal connection to me, please mention it, but it’s not necessary.)
  • My online AMC prep course is open for business. If you need to learn the fundamentals of AMC math, this is the place for you.
  • I know a lot of specialists who can help in various ways, and I see connecting people as an important (and enjoyable!) duty. If you need specific help, let me know what you’re looking for and I’ll do my best. And if you’re a fellow professional, please get in touch to let me know what you do best, so that I can help connect you with families who need it.

Other quick links

  • My buddy Amie Dorsey and I just recorded the 13th episode of our (now-monthly) podcast WATEC: The Excellent Conversation. You can just search for it on your podcast app, or watch it on YouTube. They’re about an hour each, and you’ll find links to snippets of the conversation in the comments so that you can browse for what’s most interesting to you. 
  • Wes Carroll’s Puzzler introduces parents and students alike to AMC problems and related brain-teasers. (No scratch paper required!) It’s fun for families to challenge each other around the kitchen table, and it’s also good for listening on your walk or run.

As I’m sure you know, subscribing to either or both of them lets us notify you when new episodes are ready, and also helps others to find them.

Last words

Hey, thanks for showing up here. I hope you found it helpful, or at least interesting. Subscribe to my newsletter (i.e. this) at wescarroll.substack.com.

I’m [email protected], I read all feedback personally, and I really do care whether this was helpful, and how I can make it better next time around. 

Here’s to another great school year for us all!

Learning vs education

by | Aug 24, 2020 | bespoke, Blog, college admissions, continuing education, distance learning, education advocacy, general update, SAT prep, teaching style, test prep, the value of education, Why should I care?

Today Seth Godin posted:

Education is the hustle for a credential. It exchanges compliance for certification. An institution can educate you.

Learning can’t be done to you. It is a choice and it requires active participation, not simple adherence to metrics.

Learning is the only place to find resilience, possibility and contribution, because learning is a lifelong skill that isn’t domain dependent.

Most of the learning moments in our lives are accidental or random. A situation presents itself and if we’re lucky, we learn something from it.

I agree that learning is where it’s at, which is why we’re introducing a few new offerings in the coming weeks. If you’re on our mailing list, you’ll see those shortly. (And if not, let me know, and I’ll be happy to add you.)

But I also think that if you can rack up an educational credential as part of the learning process, it’s worth going for it. And that’s why we also continue to help students prepare for standardized tests, math competitions, and classwork.

To sum up: It’s great if you join us because you want the STEM grades and the test scores. But once you’re here, it’s the learning that’ll really blow your doors off!

So… let’s get to it.

ACT & SAT: are they winners’ games?

by | Apr 21, 2019 | Blog, brilliance, SAT prep, test prep, the big leagues, victory

I didn’t expect Farnam Street to relate so directly to college admissions, but it does, and in a way that is as important as it is deep.

According to the post linked above, the outcome of a “winner’s game” is determined mostly by the actions of the winner; the outcome of a “loser’s game” is determined mostly by the actions of the loser.

So, in a winner’s game, superior skill matters, whereas in a loser’s game, it’s consistency that matters.

Everyone acts as though the SAT and ACT are winners’ games.  And, for middle-tier scorers, they probably are.  But for high scorers, these tests are loser’s games, 100%.

In other words, you don’t get from 1400 to 1550 by becoming smarter or more knowledgeable. You get there by consistently avoiding mistakes.

You show me a 1400 scorer, and I’ll show you someone who already has the necessary knowledge, but who also experiences some combination of anxiety, hubris, wandering attention, and/or magical thinking when it comes to answering the hard questions.

Getting rid of such bad habits is really hard. That’s why so few people get perfect scores.

That’s also why high scores are so prized.

Cheating on the SAT

by | Jan 9, 2017 | Blog, economics, SAT prep, Uncategorized

The thing I can’t get out of my head when I read this NYT article on kids paying other kids to take the SAT for them is simply this:

All they charged was $3600? Cripes, that’s a bargain at ten times the price.

Don’t get me wrong: you shouldn’t cheat, both because it’s wrong (which should be enough reason) and tactically too risky (in case the first argument wasn’t enough).

But I mean, come on, let’s do the math: a one-percent increase in salary over your life is easily a five-digit number even if you’re kind of a slacker. Two significantly different SAT scores mean admission to schools of two significantly different calibers. And I doubt the salary increase we’re talking about here is just 1%.

For those of you who are fans of the Drake equation, which uses best-guesses to try to figure out whether there’s intelligent life out there, I challenge you to apply this reasoning to SAT prep.

In fact, you might even try to create an analogous equation governing this stuff, like I just did. (I hope you have more luck than I did; if so, please let me know.) But, equations aside, it’s not really that hard to think about.

To figure out what a higher SAT score is worth, just do the following steps:

First, get a lifetime earnings calculator. (Google it; there are many.)

Then, use it to estimate the student’s lifetime earnings, given that he or she attends the best school to which he or she can gain admission given the initial SAT scores.

Then, take the average (expected) gain in SAT scores given a particular preparation method.

Then, use the calculator to estimate the student’s lifetime earnings, given that he or she attends the best school to which he or she can gain admission given the final (expected) SAT scores.

The difference between the two lifetime earnings is the value of the higher SAT score.

And now that I’ve said all that out loud, I’m starting to realize that “only four-digit” prices for SAT prep only make sense for providers who can offer only single-percentage-point gains with a high variance, as delineated in this article in the Wall Street Journal regarding the average benefit of SAT prep.

Now these days, I am no longer doing SAT prep, having left Bodsat Prep in 2016. However, the work I do preparing students for the AMC competitions (as well as the Math 2 and Physics SAT Subject Tests) still seems to be governed by this math.

Since we don’t often see prices like this, the conclusion I come to is: almost no one is delivering reliable results. (Or the people who are aren’t also good at pricing.) Interesting.