Goodbye, Princeton. I loved you while it lasted.

I will no longer be recommending Princeton Review’s Math 2 prep books, because their latest edition features a test that is a problem-by-problem parody of an official College Board test.

I don’t have a problem with close copies of official tests — on the contrary, that can be a very sensible strategy for creators of practice materials. However, my methods rely on a rich collection of problems that are different enough from one another that the student can come to generate underlying principles that apply broadly to many different kinds of problems.

Those methods of mine are undermined by problems that closely mimic other problems in the training corpus.

Therefore, I have to reject PR tests, because they follow a strategy that undermines mine.

(Alas, poor Yorick.)

10,000 hours?

You don’t have to make yourself miserable to be successful. Success isn’t about working hard, it’s about working smart.

Andrew Wilkinson

When it comes to homework, more is not better.

You’ve probably heard  of Outliers (“The Story of Success”), Malcolm Gladwell’s 2008 opus, based in part on research by Anders Ericsson. Even if you haven’t read it yourself, it’s probably been quoted to you. It’s the main source of the idea in popular culture that true mastery requires ten thousand hours of practice, which in turn has been used to justify all manner of craziness, including too much homework.

Let’s set the record straight: Gladwell’s book is not the source of the craziness per se. The craziness happened because somewhere between Ericsson’s original research and the reports on Gladwell’s book in the popular press, the proper context and conclusions got lost.

Gladwell (or at least those who reported on his book) popularized the idea that across a wide variety of fields, and especially in cognitively demanding ones, true mastery generally requires a minimum of ten thousand hours of a particular kind of practice.

A common misreading of this observation was the notion that since it’s the magical 10,000 number that determines successful mastery, one should get in as many hours as one can, as soon as one can. Hence, for example, never-ending homework assignments.

This sounds good, but unfortunately it isn’t what the research actually showed.

The right kind of practice is smart.

The research showed that the right kind of practice leads to improvement, and the wrong kind of practice does not.

This is true no matter how much practice you put in. However you define “mastery,” the right kind of practice will
get you there fastest. Furthermore, continued correct practice will continue to improve your skills. (The 10,000 hours number comes from the observation that when you are competing with other people, being the best depends on putting in an enormous quantity of the right kind of practice, because everyone you are competing with will be doing the same.)

Let’s get practical. How do we do this, and what else can we learn here?

The main thing is to stop practicing mindlessly. Believe it or not, practice doesn’t actually make perfect, no matter what people say. Instead, practice simply makes permanent; only perfect practice makes perfect.

So what is perfect practice? It’s a situation that challenges your current ability. So strive at every session to work ever so slightly beyond the highest level you can reach. It’s demanding, frustrating, and often annoying work. Generally, it can’t be done for more than about three hours a day. It is not generally pleasant. But it works.

Make your work better.

So remember: quality, not quantity of practice is the most important factor. Even a hundred hours of quality practice will move the needle. On the other hand, though, you could spend your whole life practicing something by rote and never get any better. (Do you hear me, teachers who assign piles of mindless work?)

Your goal shouldn’t be to do a ton of work, and your teacher’s goal shouldn’t be to pile it on. Instead, the goal should be to make your work better. The better your work, the less of it you will need, and the faster you will progress. Avoid the kind of inefficient work that doesn’t get you closer to mastery.

You don’t need more smarts

Another important lesson here is that although talent matters, it doesn’t matter much. Most of life isn’t actually a “more talent is better” situation. Rather, it’s more like “you must be at least this tall to ride this ride.” Some talent is necessary, but beyond that, more is not appreciably better. “A basketball player only has to be tall enough—and the same is true of intelligence. Intelligence has a threshold,” says Gladwell.

So, do smarts matter? Yes, but not as much as you think. You are already smart enough to be doing what you’re attempting. Believe it or not, being smarter wouldn’t help you all that much.

In pretty much any area of human endeavor, people have a tremendous capacity to improve their performance, as long as they train in the right way. If you practice something in the right way for a few hundred hours, you will almost certainly see great improvement. And from there, you can keep going and going and going, getting better and better and better. How much you improve is up to you.

Summary

All that matters from now on is how well you learn.

How to Be a Bright(er) Student: Be Your Own Pilot

Spring is the busiest time of the year for most high school students. But seniors, you face a special challenge: your support system has already done its work, and you’re on your way to college. But are you empowered to start making even more of your own decisions?  Being an effective self-advocate and self-manager is absolutely essential for success in the next phase of your life and career. Let me help you help yourself along the right path by giving you a preview of what’s to come, and some ideas about how you can grapple with it successfully.


Should you fail to pilot your own ship, don’t be surprised at what inappropriate port you find yourself docked.

Tom Robbins

 

The more decisions that you are forced to make alone, the more you are aware of your freedom to choose.

Thornton Wilder

 

A good plan violently executed now is better than a perfect plan executed next week.

George S. Patton

Getting help

As you may have noticed, making important life decisions is mostly regarded by our culture as something best left to the experts: first your parents make decisions for you, then college counselors, then graduate advisors, then professional mentors and managers, and on and on and on.

This isn’t necessarily such a bad thing: experience often leads to better results. (Also, there are certain kinds of mental tasks related to decision-making that become biologically easier in ones early 20’s, so advisors for students may be especially helpful.)

Sometimes these experts will be amazing professionals with fantastic, groundbreaking advice for you, and sometimes they’ll just be a “safety net” of decent advice, so your very worst decisions won’t be too bad.

You need to be able to tell the difference. Ultimately, you’d prefer to make your own decisions, perhaps informed by the wisdom of others, but not defined by it or by them.

Going it alone

Next, consider that most decisions are informed by your “autopilot.” You make decisions at least partly (if not entirely) by seeing that the current situation matches some past situation (possibly in relevant ways, and possibly not), and then doing in the present whatever you think was the right thing to do in the past. (See Gladwell’s Blink and Kahneman’s Thinking, Fast and Slow for more in-depth knowledge on this.) This is not a bad thing, but it’s useful to understand how to balance this important but unconscious force with your conscious, executive mind.

And even your conscious mind can have arguments with itself. Aligning your identity, your emotions, and your cognition behind a single decision can be hard work. For example, there might be disagreement between what you want, what you think you want, and what you want to want. It’s so complicated that it can be hard to get started.

But a good system can help you overcome these difficulties. Through practice, you’ll come to better understand how to align these drives within yourself. The process of making decisions will become easier. I recommend you find one you like. To get you started, I’ll provide one of my own.

Mine is a method for making decisions that helps balance your mind, heart, and gut. So it avoids a lot of the problems with “unbalanced” decision making. However, it takes a long time to get really good at it, so think of this as a long-term project. If you give this a try for now, and keep up with it, then sooner or later this method will become part of what makes you especially capable.

It works on problems big (“What should I do with my life?”) and small (“How can I solve this difficult homework problem?”). I call it the Pilot Program.

Before we get too far into it, let me acknowledge that there are many, many excellent processes out there for setting and achieving goals. There are two ways in which the Pilot Program is different from most:

  1. It uses a “ready, fire, aim” strategy. This means that you start quickly, even before you have all the details worked out, and you fine-tune everything later. This strategy helps you avoid overwhelm and perfectionism, and it helps you build momentum fast. Yet it’s also accurate over the long run, because of the fine-tuning that is built into the system.
  2. The focus of the Pilot is on constant improvement, which allows it to transcend mere goals and aim for more powerful targets: vision and values. As a result, the Pilot Program is not limited to short-term projects, but rather can be applied on a large scale—as large as you can imagine.

The five parts of the pilot program

The five parts of the Pilot Program are: a Desire, an Action List, a Journaling Plan, Course Correction, and a Finish Line.

Desire: What I want to have achieved, what I want to have, what I want to have done, or what I want to be.
Action List: Specific actions I should take in order to get closer to that vision
Journaling Plan: The location of my notes on my progress, and my schedule for reviewing the notes
Course Correction: The actions I should take in order to fine-tune all five of these parts, as I gain experience by doing them.
Finish Line: How I’ll be able to tell that I’ve succeeded.

A quick example

Here’s a quick example of how this might work:

Desire
To be a straight A student.

Action List
1. Discover what straight-A students do that is different from what I am used to doing
2. Identify the parts I can’t easily do
3. Learn how to do those things
4. Turn that learning into habits

Journaling Plan
Every week, I write down in an online journal what I’ve done during the week that is likely to raise my grades.

Course Correction
Every week after journaling, I read through the whole journal from the beginning, and then I do two things: (1) make any changes to the Desire, Journaling Plan, action list, Course Correction, or finish line, according to my best judgment at the time, and (2) write notes telling myself what I should try to do the following week in order to stay in alignment with those parts.

Finish Line
I have a report card containing nothing but A’s for the current grading period.

A life-size question

Now let’s say you’re trying to figure out what to do with your life. That’s an awfully big question! But the size of the question isn’t a problem. Here’s how it might work:

Desire
To know what I want to do with my life.

Action List
1. Ask some older people what they do, and whether they like it
2. Think about what they say and how it applies to me

Journaling Plan
I record all the interviews, and after each one I listen to it again and take notes, focusing on how I think the ideas in that interview apply to me (or not).

Course Correction
I think about which interviews seem most and least helpful, and I use that information to determine what sort of person to try to interview next.

Finish Line
I have written down a specific “life Goal” that I believe is worth pursuing.

A research project

Here’s a different way in which someone might use the Program to come up with their Desire:

Desire
To know what I want to do with my life.

Action List
1. Ask myself what I want to do
2. Pay attention to what I find myself saying

Journaling Plan
Every time I ask myself the question, I come up with a one-sentence answer and write it down on a piece of paper taped to my wall.

Course Correction
Once a month, I spend ten minutes reading the paper and remembering what made me give each of those answers.

Finish Line
I have noticed myself giving the same answer three or more times.


As you can see, two different people could use this method to solve the same problem, and wind up with very different paths and very different results.

Let me be clear: the method isn’t easy. You have to really think about each of the parts, and that takes time. Sometimes, it also requires courage. But the personal attention you put into it is also a major strength of the Program. Since the parts all come from you, they aren’t nearly as vulnerable to anyone else’s bias. You are making all the decisions. You are also deciding the very rules of the game. This means that your decisions using this method are more likely to have high integrity. In other words, true to yourself.

So you get to decide how much “outside bias” you’re going to let in to your process. Let me suggest some guidelines for deciding how much bias to let in, and how to avoid that bias when you want to.

How much bias

A good guideline is replicability: if many people before you have gotten what you want, and it’s clear how they did it, it’s probably best to let others’ guidance help you. This can mean researching how to do it, or trusting an expert to help you find the best path. Note: this doesn’t mean to relinquish all control (or worse, responsibility for the outcome); it just means that you’ll probably do better with advice than without.

By contrast, if the result has not been replicated many times, or if it isn’t clear how it’s been done in the past, then you may be the best guide for your own process, because (1) you care more than anyone else, and (2) you probably know yourself better than anyone else. (One key exception: one or both of your parents might be even better guides than you yourself are.)

Another good guideline is boundedness: problems and questions that are well-defined, short-term, multiple-choice, and/or straightforward are usually best answered with outside help or bias. However, more nebulous, long-term, open-ended and difficult-to-even-define problems and questions usually benefit from removing as much bias as possible from the decision process.

One last thing: it’s okay to use one method now and another later. For example, many people enlist the help of a college counselor (high bias, high expertise) to figure out where to go to college, and only later use their own process (low bias, high integrity) to determine what sort of career to build on the foundation of the college career.

How to avoid bias

Read through each line of your analysis. Ask yourself: why is this here? Whenever you have a hard time answering that question, find and remove the bias.

How bias can help

The form of the question often gives you some idea about what kinds of answers would be good or bad answers, and this can help you, especially when you’re just getting started.

For most problems, there are solutions that are considered “out of bounds.” That is: there’s a strong sense of what you are “allowed” and “not allowed” to do. For example, take the SAT. There, it is not considered valid to write in your own answer choice on the multiple-choice sections. This means that most problems carry with them implicit hints about how the problems should be solved. As you learn to recognize bias, you will learn to recognize solution “hints.”

One other way in which bias can help is that it can show you what everyone else has tried, so that you can try doing the opposite. Some very successful companies, for example, got their starts by trying to do what was considered at the time to be “obviously” impossible.

How eliminating bias can help

At its most extreme, the Pilot Program helps you to examine all the possibilities. Even the “crazy” and “impossible” ones. So make sure you take advantage of that leeway by including all the “outside the box” possibilities in your analysis.


Summary

Making your own decisions is actually a lot harder than it sounds. A reliable method helps. Here’s one such method. Getting good at it really pays off.

Expert level

Once you get comfortable doing this, remove the explicit Desire and Finish Line from a project, and continue only with the Action List, Journaling Plan, and Course Correction.

You’ll often find that an area of self-improvement is made much more powerful by removing the “fixed destinations” from your thinking and framework, and instead, putting your focus on open-ended, continuous improvement. This is related to the Mindsets described in Chapter Five.

Creating the habit of continual improvement, where it’s about always taking the next step (Action List, Journaling Plan, Course Correction) is one of the most powerful meta-habits you can build.

What’s wrong with math education

“Math punk” Tom Henderson has written a brilliant essay that I have co-opted and edited here.

In a nutshell, what is the problem with math education in the US? It’s that students are mostly trying to minimize feeling stupid rather than trying to maximize their ability to solve problems.

This manifests as “show me The Steps.”

Many students want a sequence of steps that they can perform that will give them an answer. This is not unreasonable; they know that their performance on exams, and therefore their performance on the All-Important Grade Point Average, is largely determined by being able to Do The Steps. So they want to know the formulas, so that they can float them on top of their short-term memory, ace the exam, and then skim them off.

For their entire mathematical careers, math has been a sequence of Steps, and if they get them wrong, they get red pen, bad grades, No No No Look What You Did. Plus, bonus, there is no apparent relevance of these algorithms other than To Get The Answer.

But that’s crap. “The Steps” aren’t math, and what’s more, The Steps aren’t generally useful in life.  What’s useful is the ability to deconstruct thorny problems and figure out a way to tackle each of the pieces.

The Steps are seeing the sorts of symbols that count as “right”, and trying to replicate that dance of steps. It turns out that the easiest thing in the world is to look at a student’s work, and tell the difference between “Knows what’s going on, made mistakes and dozed off” vs. “Can memorize steps, has no idea what’s going on.”

Now, a better way to explain mathematics sort of looks crazy at first. It’s handwaving. It’s referring to certain groupings of symbols as “alphabet soup” and writing it down as a wild scribble with one or two symbols around it.  It looks nothing like standard “math class” from the outside.

That’s because the better way avoids showing The Steps and instead shows enough of The Idea that the student can reconstruct what the steps MUST be.

And that brings us to a better way to learn mathematics: you get a fear-free zone, you check your ego at the door, you try a bunch of things that will wind up not working, you ask a pile of dumb questions, and before long, you figure out some crazy way to get the problem solved.  And only then do you realize that your crazy, lame-brained, that-can’t-possibly-work solution is in point of fact the official method nine times out of ten. Because math is, at its core, just a collection of “the best way we could figure it out” stories, organized semi-sensibly, and with a specialized vocabulary and language on top.

So, what’s wrong with math education in the US? What’s wrong is: whatever it is that makes students uninterested in learning any more math than is required to minimize feeling stupid.  My solution?  Provide that safe space; find the genius in every question; and provide interesting problems to solve.

Ultimately, that’s all it takes to get students to say “Oh. That’s not nearly as hard as I thought it was going to be.”

Put a little math in your life!

Brian Greene makes the point about science, and it holds just as true for math: “We rob science education of life when we focus solely on results and seek to train students to solve problems and recite facts without a commensurate emphasis on transporting them out beyond the stars.”

Interested in communicating with a faraway friend without allowing anyone to eavesdrop on you? Of course you are; this problem affects a middle-schooler’s daily life, and yet it is also the basis for modern commerce: communicating your credit card number over the Net without allowing a thief to eavesdrop is a non-negotiable requirement for our economy. How is this problem solved? First, you try for a while. Then let’s talk about prime numbers and see what they can do for us.

Interested in taking a rocket to the moon? Well, if you want to drive a spaceship, you probably ought to understand how gravity works differently from your intuition when you’re far from Earth. Let’s talk about ellipses, and while we’re at it, let’s predict the next approach of Halley’s comet. (And if you’re a high-school sophomore or so, we can talk about inverse-square laws while we’re at it.)

What if we picked up a signal for an alien civilization on our radio telescopes tomorrow? Do you think they’d speak English? Not a chance. Let’s think about how we would communicate without a shared language, a shared culture, even a shared planet? Well, there’s one thing we have in common with any race advanced enough to send us a signal like that: they know math. Let’s think about how that might work…

One of the biggest rarely-asked questions in math is simply “why should I care?”

I love answering that one.

What’s wrong with standardized tests for teachers?

Well, I really mean to answer “what’s wrong with using the results of students’ standardized tests to evaluate teacher performance?”

On its surface, nothing. Indeed, student performance should be the primary metric of teacher quality. When I worked for Bodsat Prep, I was gauged by that metric myself: what matters is the point gain.

The main problem, though, is that the standardized tests don’t test everything that we want our students to know. In the same way that the standardized test at the DMV doesn’t tell you whether you’re a good driver (though it does catch some really bad drivers), standardized testing for schools doesn’t tell you whether you’re on track to be a productive member of society (though it does catch a number of people who really aren’t).

Also, if a standardized test samples say a random 10% of what one should know, then how well one does on such a test is, statistically, a good indicator of how much of the target material one knows. But if it’s a non-random 10%, then you can count on prepared students to know that 10% very well. And possibly nothing else.