What I wish I knew in high school

• Math is awesome. I thought I knew what math was. After all, I’d been taking it for my whole life. But there’s so much more math than what you learn in high school! Number theory, game theory, probability, algorithms, topology…

In high school, I liked math and wanted to learn more math, but the guidance department told me that if I learned anything on my own I’d just be required to repeat it in school. But you can learn math that’s different from your curriculum, and it’s usually more interesting. Check out the Art of Problem Solving books, which help you prepare for math competitions. You can take an online math course from a great university (see here), such as Stanford’s “Introduction to Mathematical Thinking”. You can apply to go to Math Camp, which I’ve heard is awesome.

• Yes, you can learn to program! I heard my high school doesn’t offer AP Computer Science anymore. That’s too bad. However, most of the people I know at MIT who knew how to program before college taught themselves because they wanted to create a specific project. All of the ones who are good at it learned a lot outside school. Most languages have great online documentation. Think of something you want to create, pick a language (I suggest Python), and do it!

• Science, technology, engineering, and mathematics (STEM) majors have a much easier time getting a job and make more money. But they generally have to work harder.

• The liberal arts scam. When I was applying to college, I heard over and over that a “liberal arts education” gives you a broad education in almost everything, develops your thinking skills, and makes you an appealing job candidate. First, check out this Forbes article on unemployment rates for different majors. And remember that if you follow your heart and do one of those high-unemployment-rate majors, you’re more likely to have to take a job that has nothing to do with your major.

Liberal arts advocates draw a dichotomy between “developing specific skills for a specific job” and “gaining broad problem-solving abilities”, and claim that a liberal arts education develops the latter (and implicitly, other programs don’t). However, it’s not true that a liberal arts education always makes you a better problem-solver, and it’s definitely not true that technical programs just prepare you for some specific job. It depends on the college and, mostly, it depends on you.

Take MIT, for example. Students must take eight humanities, arts, and social sciences (HASS) classes, distributed over different subject areas. Two of these must be writing-intensive, and they’re all hard. In addition, we need to take two communication-intensive courses in our majors. And our majors don’t just prepare us for a single job, but rather give us problem-solving techniques that are widely applicable. For example, learning to express yourself in the language of mathematics or programming forces you to be clear, logical, and consistent. No bullshitting allowed. On the other hand, many liberal arts programs demand that you take a wide variety of subjects, but it’s possible to take classes that are really easy, or so narrow as to be irrelevant. The “science for non-science majors” classes that students often take to fulfill requirements are generally a joke. In general, STEM majors have to know much more about the humanities than humanities majors have to know about math and science.

Basically, whether you learn to reason and communicate about a wide variety of subjects has more to do with what classes you choose to take than with what college you go to. But for goodness’ sake, take classes that will make you smarter and help you get a job, or you’ll regret it senior year. And unless you’re at a school like Harvard, don’t believe anyone who tells you that if you pursue your passion and major in creative writing or theater or anthropology, a job will surely follow.

• Take time to pick your major. Many people think they know what they want to major in, then change their minds when they discover something cooler. This can often result in graduating late. I recommend exploring many different classes in different majors early, then specializing only once you’re sure about want to major in. Some colleges require you to declare a major much earlier than others, and some require students to take so many classes in their major that they can’t afford to explore. Look into this before you decide where to go.

A lot of people pick a major that just continues what they did in high school. I made that mistake when I declared a physics major, then later discovered that I was much more interested in computer science and economics.

Genesis Chapters 1-5

I thought I knew the Eden story. God makes man, tells him not to eat from a certain tree, makes Eve, snake tricks Eve into eating from that tree, etc. Except something isn’t quite right about that…

2:16-17 And the Lord God commanded the man, saying, “You may surely eat of every tree of the garden, but of the tree of the knowledge of good and evil you shall not eat, for in the day that you eat of it you shall surely die.”

3:4 But the serpent said to the woman, “You will not surely die. For God knows that when you eat of it your eyes will be opened, and you will be like God, knowing good and evil.”

3:7 [After eating from the tree] Then the eyes of both were opened, and they knew that they were naked. And they sewed fig leaves together and made themselves loincloths.

3:13 Then the Lord God said to the woman, “What is this that you have done?” The woman said, “The serpent deceived me and I ate.”

3:22-23 Then the Lord God said, “Behold, the man has become like one of us in knowing good and evil, Now, lest he reach out his hand and take also of the tree of life and eat, and live forever -” therefore the Lord God sent him out from the garden of Eden.

Contra 3:13, the serpent didn’t deceive Eve. God did! The serpent told the truth: Adam and Eve didn’t die that day, but rather gained knowledge of good and evil.

Adam and Eve, not Adam and Steve

The beginning of Genesis also has some points to make about gender roles. 

2:24 “Therefore a man shall leave his father and his mother and hold fast to his wife, and they shall become one flesh.”
3:16 To the woman he said, “I will surely multiply your pain in childbearing; in pain you shall bring forth children. Your desire shall be for your husband, and he shall rule over you.” 

Strong words! I once heard an Episcopalian priest say that the Bible is more descriptive than prescriptive. When I read this I thought, wait, those verses are really prescriptive. Every man goes with a wife, every woman is ruled over by a husband. But the way these verses are phrased  sounds like more of a description of the human condition than a prescription of how we should live. It doesn’t sound like “Women, if you have the option of being ruled over by a husband you should take it.” It sounds like “You will never have that choice.” 

Bible blogging

When I was in middle school and high school, I tried really, really hard to read the Bible, but  my ADHD made it really hard to read anything at that point. I didn’t get very far. I’m going to try again, but with the semester approaching I don’t expect to get through very much of it.

Anyway, I am going to highlight things that surprise me or that I hadn’t noticed before. That is, the interesting parts. I am not going to rehash the parts that are familiar to most Christians, and I am not going to highlight things that don’t make sense for the sake of scoring points again the Bible, unless there is an issue that is new and interesting to me.

Feedback is always appreciated, but especially so on this topic I don’t know much about! I’d like to investigate other religions next, so if you think your religion has an awesome holy text I’d love to hear about it.

Don’t Abuse Statistics 101

Jezebel cites the blog Tits and Sass‘s discussion of a study showing that strippers make more money while ovulating.

The first time I heard about the ‘Ovulating Strippers Make More Money!‘ study, I thought, “oh, that’s interesting! I wonder what they learned. So many strippers must have been studied.” But NO. Here’s the story: Two researchers at UNM in Albuquerque studied dancers for two months. They studied 18 dancers. In one club, in one city. For two cycles.

Two!?! I thought everyone knew that three makes a trend.

I’d like to dispense with the notion that small sample sizes invalidate a study.¹   It’s true that small samples make it harder to prove anything, and if you are able you should study as many people over as long a time as you can. These researchers had only 18 women and 36 menstrual cycles to study (and 296 work shifts), but they found a large enough effect that they were able to confidently state that there was an effect. I’ll explain the connection between sample sizes, measured effect sizes, and statistical significance.

Statistical studies typically include a null hypothesis, and a p-value. The null hypothesis is a statement that there is “no effect”: In this case, that ovulation has no effect on lap dancers’ earnings. This study says they make about $90 more per shift when ovulating than when they are in the luteal phase (not ovulating and not menstruating). The p-value answers the question, “If ovulation has no effect on earnings, how likely is it that we’d measure an effect as high as $90?” In this study, the p-value is less than 0.1%: If a similar study were run 1,000 times and ovulation had no effect on earnings, you would expect to find an effect as large as $90 just once. Small p-values are good.

Small sample sizes (using a short time period, or a small number of strippers) mean that you are likely to find results that lie far from the true value. For example, suppose you know the true effect of ovulation on earnings is $0, and you study three women for one month and find that they made an average of $10 more while ovulating. That shouldn’t surprise you, because it’s very likely that random chance caused the difference. But if you interview 1,000 women over ten years and find out that they average $10 more, it’s unlikely that random chance caused the difference.

But the fact that small samples are less reliable is incorporated into the p-value. The study with three women would have a much larger p-value than the study with 1,000, even though they both measured a $10 change. Equivalently, the small study would need to measure a much larger effect to have the same p-value.

Let’s walk through an example.

Say you have a bucket of red and blue balls. You want to test whether there are more red balls than blue balls. The null hypothesis is that there are the same number of red balls and blue balls.²

You draw one random ball, and put it back. It’s red. If the null hypothesis (called H₀) were true, this would happen 50% of the time you drew one ball. The p-value is 0.5.

For your next trial, you draw and replace five balls. They’re all red! If the null hypothesis were true this would only happen in 3.1% of five-ball trials. Conventionally, this means you’ve attained statistical significance: The p-value is 3.1%, and 5% is usually considered “significant”. Since you would only draw this many red balls in 3 of 100 trials if half the balls were red, there must be more red balls. (This conventional “frequentist” approach has a lot of problems, which I’ll cover next post.)

Now say you draw and replace eleven balls. If 11 of 11 are red, the p-value is 0.05%, or 1 in 2048. If 9 of 11 are red, the p-value is 3.3%. This means that if you repeated the 11-ball trial 100 times, you would only expect to get 9 or more red balls in 3 of the trials.

If you draw 20 balls and 15 are red, the p-value is 2.1%.

With 5 draws, you need 100% to be red to “prove” that most of the balls are red. With 11 draws, you need 82% to be red. With 20 draws, you need 75%, and with 100, you need only 60%.

Journals will generally only publish studies with a p-value below 5%; a low p-value implies a low probability of false positives, or finding an effect when none is there. Small sample sizes create a high probability of false negatives, in which an effect is there but you can’t find it. So, if you have a study with a small sample size and a “statistically significant” finding, you can’t say “Oh, they only found something because there is such a small sample size.” If you have a study with a small sample and no statistically significant finding, you can say “Just because they didn’t find anything doesn’t mean it isn’t there.”

¹ In this case, it’s true that if the dancers’ cycles were synchronized because they all worked together, and ovulation coincided with a time of good business, this would invalidate the study. But they didn’t actually all work at the same club; the researchers advertised all over the city and got respondents from multiple clubs, although the paper doesn’t specify how many.

² Technically, the null hypothesis is x=0.5 , not x<=0.5.

What I read in 2012

What should I read next year? Here are most of the books I read this year, all of which I recommend.

Poor Economics: A Radical Rethinking of the Way to Fight Global Poverty, by Abhijit Banerjee and Esther Duflo. These two economists study poverty mainly through randomized, controlled trials. Some methods for fighting poverty are very effective, some aren’t, and Westerners’ assumptions about the lives of the world’s poor and what can help them are often wrong. This book reviews what we know about poverty at the family level  the importance or lack of importance of hunger, health, education, and family size) and then at the level of institutions (credit markets and savings mechanisms, microfinance, entrepreneurship, and politics). This book is pretty awesome. There are few broad conclusions but lots of great ideas and evidence.

Why Nations Fail: The Origins of Power, Prosperity, and Poverty, by Daron Acemoglu and James Robinson. The authors argue that good or bad – “inclusive” or “extractive” – economic institutions account for most of why a society is wealthy or not, and that whether a society has inclusive or extractive political institutions determines whether it has inclusive or extractive economic institutions. It’s compelling, but the arguments get repetitive. I only got about two-thirds of the way through. I took a class called “Political Economy and Economic Development” last semester, and noticed that many of the studies cited in Why Nations Fail were assigned as readings. They are generally very readable, so if you have a statistics background I’d suggest just reading them and skipping the book.

The Conservative Soul: The Politics of Human Difference, by Andrew Sullivan. I read this book a while ago and don’t remember it clearly. Sullivan describes a “conservatism of doubt”, in which we should be cautious about implementing new policies but absolutist about very few of them. It’s a good idea, but most of the book is about what Sullivan’s conservatism isn’t: It isn’t based on faith, or natural law; he’s not a neoconservative, or a fan of George W. Bush.

Drugs – Without the Hot Air: Minimising the harms of legal and illegal drugs, by David Nutt. This book is based on solid evidence, and had some real surprises for me. Marijuana is more harmful than I thought it was, and unlike almost any other drug, alcohol harms almost your entire body. And as long as you take it with a comforting “set and setting”, LSD is not harmful at all, and sometimes has large benefits. But I have to nitpick: quantitative isn’t the same thing as scientific. Asking scientists to make up a number to describe how harmful a drug is and averaging those numbers isn’t a “scientific” estimate of the drug’s harm, and isn’t that useful.

The Marriage-Go-Round, by Andrew Cherlin. I liked this book. American families are more tumultuous than in the rest of the world: we marry more, divorce more, and move in with partners more frequently. Cherlin claims that this tumult is just fine for adults who choose it, but bad for children. He says we should be slower to marry and slower to divorce. Against the conventional wisdom and worth a read.

Harry Potter and the Methods of Rationality, by Eliezer Yudkowsky. It’s Harry Potter fan fiction, yes, but it’s full of interesting ideas and a great story.

Thinking, Fast and Slow, by Daniel Kahneman. I’m currently reading this. It is basically about the psychology of making decisions, with a focus on where we tend to get things wrong. I agree with all the rave reviews of this book you can find on the internet.

The Fellowship of the Ring, by J.R.R. Tolkien. I’ve read it before, of course, but it still gives me the shivers, and the warm and fuzzies.

Combating Overconfidence

In my last post I discussed some very common cognitive biases: overconfidence, in which you feel more certain of your beliefs than you should, and confirmation bias, in which evidence favoring your position looks stronger than evidence opposing it. I think these tips have helped me become less overconfident. Of course it’s hard to tell how overconfident one is, but I change my mind more often than I used to,  frequently come to conclusions I wouldn’t have expected at the beginning of a quest for information, and often change my mind. I think this is decent evidence that I have become less overconfident.

So here are some tips.

• 0: Take probability and statistics classes, and be able to understand academic research in the topic you are interested in. Many people who discuss cognitive biases say the way to counter them is to become more aware of them. For example, people conclude far too much based on small sample sizes – an account of how a policy change affected one person, or one town – and they also fail to separate correlation from causation. Don’t just become aware of the limitations of small samples, or of “correlation bias”: seek out higher-quality information. Whatever you’re arguing about, someone has probably spent many years studying it. Unfortunately, understanding much research requires understanding statistical methodology.

In my high school’s AP US Government class, we were frequently assigned to write a short paper on an unfamiliar topic, and then discuss it with our classmates. This is great training for learning to communicate and persuade, but it’s not great training for learning the truth. Say we had to discuss whether the minimum wage should be raised, or should even exist. “Mary” would talk about how maximal freedom leads to the best outcomes, and how government’s efforts to intervene nearly always go awry. “Jim” would talk about the duty to protect the neediest. “Andy” would say that if you think about it, it’s clear that minimum wages increase unemployment, and hurt the people they are designed to help. “Carla” would say that studies show minimum wage laws help the economy by putting more spending money in people’s pockets, and actually increase GDP. Along the way, all these clever students do a great job countering their classmates’ arguments. The ones who have taken AP English Language and Composition are especially adept at sniffing out a bad argument and identifying fallacies.

But none of this really decide whether (1) minimum wage laws are good or bad, or (2) if they’re good, whether the current level of the minimum wage is the correct one. To answer (1), you’d want to know

1. Do minimum wage laws increase unemployment? By how much?
2. If so, whom do minimum wages hurt? Do they reduce employment among low-skill workers, who may become permanently unemployed? Or among teenagers?
3. How many people get wage increases as a result of minimum wage increases? Which people? Who is making the minimum wage now?
4. Do minimum wage laws help GDP growth by giving more money to low-income workers, who spend all they get? Or do they hurt GDP growth by hurting businesses’ profitability and increasing unemployment?

Evaluating (2) is even harder; you need to know how the magnitude of these different effects varies with the level of the minimum wage, and whether raising it from its current level would cause more good than harm. This answer should change depending on the current minimum wage.

Economists have extensively studied all of these issues. Some studies are more credible than others. (Maybe this was a bad choice of issue – it turns out that economists are split on minimum wages.) However, high school students don’t have the ability to read or evaluate these studies. Few can give a quantitative estimate of the effect of minimum wage increases on unemployment. None can read Card and Krueger’s meta-analysis and understand why t-statistics clustered around 2 indicate publication bias. The answers are out there, but outside the reach of many skilled arguers.

• 1: Ask specific questions before investigating a topic, and formulate a plan. Don’t set out to evaluate a vague question like “minimum wage: good or bad”? Instead, decide what specific questions you need to answer in order to decide how you feel about minimum wages. Think about what sort of evidence you should accept, and what sort of evidence you should judge “not enough”. Decide how you will agglomerate the different evidence you find: If minimum wages increase unemployment slightly, does this mean that they are definitely bad? Or can other benefits outweigh a hit to employment? This process helps stop you from making up your mind early in the research process, and abandoning the search interpreting the evidence you find later on in a way that bolsters an early conclusion.

A while ago, I informally used a similar process to figure out what I thought about whether abortion should be illegal.

1. Does banning abortion actually decrease abortion rates? If no, there is no good reason to ban abortion. Time-series studies of countries that have changed their abortion laws would be the best way to decide this; theory gets us nowhere.
2. Does abortion actually harm women by causing serious mental health issues? If so, there is no good reason for abortion to be legal. Studies that compare women who have and haven’t had abortions and control for variables like income and previous mental health history are the best evidence I’m likely to find, but are likely to be statistically biased by omission of confounding variables.
3.  How should one weight the value of a fetus’s life? If killing a fetus is morally equivalent to killing a human adult, abortion obviously should be banned (unless that won’t change abortion rates). If killing a fetus is morally equal to killing a squirrel, abortion should be fully legal (unless it actually hurts the women it should help). Theoretical arguments are the best I’m likely to find here, but studying the development of pain and awareness could help.
4. In a modern developed country, how would banning abortion impact its safety? I actually just thought of that one and haven’t investigated it yet.

I decided all of this before attempting to decide what I thought of legal abortion. When I am not yet committed to any belief, it’s easy to decide what I should consider and how I should evaluate. It turns out that (1) legalizing abortion does increase abortion rates, so banning abortion most likely decreases abortion rates; (2) Abortion does help the women who get abortions, and the children and others they often support, with little harm to mental or physical health; (3) Science and clever arguments haven’t convinced me either way; and (4) I have no idea.

So, there we go! I have no idea whether abortion should be legal, although (3) points me towards thinking I shouldn’t get one myself

• 2: Keep in mind why you believe what you believe. If you find evidence that contradicts your beliefs, it’s hard and unpleasant to evaluate how this evidence should weigh against your previous beliefs, but it’s easier to figure out how the new evidence compares to your previous evidence.

Harry Potter and the Methods of Rationality provides a good example of this. (Very mild spoilers.) Harry believes that Voldemort is/was not very smart, and cites the Dark Mark as evidence for this. When he finds out that the Dark Mark doesn’t work how he thought it did, he keeps arguing that Voldemort isn’t very smart. He comes up with new arguments, and they sound plausible. But then he realizes that the only reason he thought Voldemort wasn’t smart was how the Dark Mark worked, and this evidence has been contradicted. He needs to regather his facts and start over, not attempt to hold onto his previous position.

• 3: Acknowledge that you don’t know. I find it helpful to state what I believe and why, and acknowledge what I don’t know. “I think Voldemort is dumb because the Dark Mark seems like a stupid design, but I don’t know much of the history of the last war.” “I would guess that the smartest students go to Caltech because they have the highest SAT scores, but maybe other colleges select students based on other measures of smartness.”

It’s hard to walk back on a belief after you’ve told your friends “Voldemort is dumb!” or “All the smartest kids go to Caltech!” Society generally rewards you for being consistent. And admitting you’re wrong is embarrassing  You can make it easier to change your mind by acknowledging upfront that you may be wrong. If this feels dishonest, keep in mind that you are more likely to be wrong than you think.

Cognitive Biases

My brain, like yours, isn’t up to figuring out many things. I’ve knit several pairs of socks, and I know how sock-making works, but I don’t know how much yarn my next pair will take. I can’t remember all the real analysis proofs I learned, and sometimes when I try to write one it looks sensible to me but is wrong. I have no idea what tomorrow’s weather will look like.

On some sorts of questions, people are systematically wrong. They consistently underestimate how long it will take to finish a project, or how much the project will cost. Most people believe they are better than average at simple tasks, like driving, and worse than average at difficult tasks, like unicycle riding. Rhyming statements sound more true than non-rhyming statements with equivalent meaning. Systematic errors in judgment are known as cognitive biases; I just described the planning fallacy, one whose name I’m unsure of, and the rhyme-as-reason effect. 

Before you read any farther, make sure you understand that these biases effect everyone, or almost everyone, and that includes you. Learning about cognitive biases shouldn’t help you win arguments – shouting “planning fallacy!” doesn’t prove an optimistic plan wrong – but they can help improve your own mind. If you need convincing, try reading Judgment Under Uncertainty: Heuristics and Biases, by Amos Tversky and Daniel Kahneman; it will walk you through some examples that almost everyone seems to get wrong. I’m currently reading Thinking, Fast and Slow, a book on the same topic, and it’s great. And check out this post.

So, there are a couple biases that are really destructive for making decisions, and especially in political judgments. This is why your crazy friends won’t change their minds in the face of the perfectly good evidence you keep telling them, and why they probably think the same thing about you.

• Overconfidence: People tend to think their beliefs are much accurate than they are. For example, if test subjects are asked a question like how many surgeons are in the Boston Yellow Pages, they will, of course, usually be wrong, and be aware that they are probably wrong. Then they are asked to give an interval such that they were 98% sure that the true number of surgeons were inside this interval. If they were “calibrating their estimates” correctly, the true number of surgeons would only fall outside their confidence intervals 2% of the time. Instead, the number of surgeons was outside their confidence interval 46% of the time. (link) This sort of finding has been widely replicated; people seem to be overconfident about a wide variety of estimates.

It’s hard to measure overconfidence in beliefs that are non-numerical, but I think it can be empirically observed. For example, polls showed that 50% of likely voters thought Mitt Romney would do a better job than Barack Obama at improving the economy, 44% thought Obama would do better, and only 6% were unsure. I don’t know what the correct answer was, but I do know that at least 44% were wrong! (Unless they disagree on what “improving the economy” means.) In general, people often have strong and opposing opinions on political questions, and this means that a large proportion of the population are wrong, while ardently believing they are right.

• Confirmation Bias: Once you believe something, you will see information that agrees with your point of view more favorably than information that disagrees. Once you believe your next knitting project should be socks, not a sweater, the pro-sock arguments you see in the sock-vs-sweater wars will all look more plausible than they should. This sometimes makes sense: if you believe the Eiffel Tower is tall, and you find an article explaining why it is so tall and another one explaining why it is so short, it’s sensible to think that the first article is more trustworthy. The problem is, confirmation bias reinforces incorrect beliefs as well as correct beliefs – and you’re more likely to be wrong than you think. And today, with partisan cable news and a variety of sources on the internet, you’re likely to seek out information that will confirm the beliefs you already hold.

I’ve found a few ways of avoiding these biases… tune in next blog post to find out!