Numbers Mean Things

So I saw this headline in The Washington Post:

UN envoy says $5 billion malaria fight has saved several thousand lives in recent years

My first thought was, "$5 billion divided by, let's say 5000 people, that comes out to a million bucks per person saved. A noble result, to be sure, but isn't there a more cost-effective way of achieving the same result?"

Then I read the first paragraph:

UNITED NATIONS — The U.N. chief’s envoy for malaria says a $5 billion campaign has saved several hundred thousand lives in recent years, keeping international efforts on track to virtually end deaths from the mosquito-borne disease by 2015.

(emphasis added) and now the cost per person drops two orders of magnitude, from $1 million to $10,000. Much more reasonable (though it'd still be nice if it were even cheaper).

But I suspect that either the reporter, or someone at AP or WaPo decided that the word "hundred" didn't change the meaning enough to make it worth taking up valuable headline space. I'm sorry, but it does.

Then again, what's two orders of magnitude among friends?

The Onion Nails It Again

From The Onion:

'Ghost Hunters' Enjoys Surprising 100% Success Rate

LOS ANGELES—Despite the fact that in all of human history not one person has ever provided definitive empirical evidence of the afterlife, the crew of the second-tier cable network television show Ghost Hunters has shocked statisticians and theologians alike with an incredible and uncanny 100 percent success rate in documenting proof of spiritual presences from beyond the grave. "Well, we have really good equipment," said one cast member, a man in his 30s who sees one or more ghosts each week, without fail, every single time he sets out to do so. "What can I say? We're just really good researchers, I guess." At press time, despite having repeatedly resolved the most central question of human existence, the program is somehow not on the cover of every major newspaper, magazine, and scientific journal in the world.

They got several things right: for one thing, the "really good equipment" used by the ghost hunters picks up phenomena that no other equipment does. That is, there's no independent confirmation that what it's picking up is real. So how do we know that the cause of the reading is a ghost or other otherworldly phenomenon, rather than noise in the equipment, or interference from a nearby radio station, or something like that?

An instrument that detects something that isn't there (false positive) is just as wrong as one that fails to detect something that is there (false negative). If a TSA body scanner finds a knife on you and you aren't carrying one, that's a false-positive. But the TSA inspectors don't know that; all they know is that they've detected a knife. They don't know whether it's a true positive (you're carrying a knife) or a false positive (their equipment misfired). So they use independent tests, such as a patdown (which depends on the knife's tactile properties, rather than its electrical properties) to check. What properties do ghosts have, that would allow us to confirm the reading on one instrument with another?

The other point is that if there's an afterlife, and we can communicate with the dead, there are huge consequences. For one thing, it would affect how we live our lives: what do we need to do to ensure a pleasant afterlife, rather than an unpleasant one?

Or think of the boon to history and anthropology, if historians could interview George Washington's soldiers, or the people who crossed the Bering Strait to colonize Alaska, or even ask William Shakespeare whether he wrote his plays.

Heck, how about subpoenaing the ghost of Ken Lay to question him about his role in the Enron scandal? Or what if a network administrator at company A passes away? Can company B hold a seance and convince his ghost to give up the network passwords?

One idea popular among conspiracy theorists is that Lee Harvey Oswald was murdered so that he wouldn't reveal details about the JFK assassination. But if his ghost is out there somewhere, maybe we could ask him.

In short, if ghosts exist and we can talk to them, why are ghost hunters making millions (let's be generous) on a cable show, rather than making billions or more with an afterlife telecom?

But the Onion said it in a funnier way.

The Economic Argument

Today's XKCD is beautiful.

The Science Game

At the end of Wednesday's meeting, I conducted a psychological experiment on those attendees who didn't need to rush off for other appointments.

It was a variation on the game of Twenty Questions. But instead of a person or thing, I'd think of a rule or category, which the players had to guess. And instead of trying to guess directly what category I had in mind, the players would call out a specific example, and I'd tell them whether it was in the category I was thinking of. Actually, I asked them to name both the category they were thinking of, and a specific example; but I'd only tell them whether the specific example fit my category, not whether the category they were thinking of was correct.

For example, I might think of the category "types of cake". A player might say, "the category is vegetables, and the example is carrot". I would then say yes, because carrot cake is a type of cake. The next player knows that "carrot" matches the rule, and might say "orange things, and the example is red hair". To which I'd say no, because there's no such thing as red hair cake.

The reason I called this the Science Game in the title is that this has similarities to the way science is conducted: you might think that light and heavy objects fall at the same rate, but you can't ask nature that directly: you can set up a mechanism that starts two timers while simultaneously dropping a 10-kg weight and a ping-pong ball from a three-meter height onto two nylon tripwires that stop their respective timers; you can do this at 9:48 on a Tuesday in room 1205 of the Schumacher Building, after which either the two timers will read the same elapsed time, or they won't.

Nature doesn't answer broad questions, only narrow, specific ones. We have to work out the general rules from the answers we get to specific questions.

At any rate, we played a round of this game, with the results given in the following table. Feel free to play along (answer below the table). To get everyone started, I provided two examples that matched the rule I had in mind: "2, 4, 6" and "3, 6, 9".

Example	Guessed Rule	Matches?
2, 4, 6	(Starter example)	Yes
3, 6, 9	(Starter example)	Yes
4, 8, 12	1, 2, 3 multiplied by some integer	Yes
1, 2, 4	Three different numbers	Yes
-2, √3, π	Any three real numbers	No
5 12, 48	Any set of numbers with exactly one prime	Yes
6, 2, 4	Any three numbers	No
12000, 13009, 4.8×1028	Three increasing numbers	Yes
1.5, 2.5, 3.5	Any three positive numbers	No
-1, 1, 2	Three integers	Yes
2, 1, -1	Three integers (again)	No
1, 2, 2	Three non-decreasing integers	No

The rule I had in mind was "a series of increasing integers". As you can see, toward the end the players got pretty close, though they didn't quite nail it.

I said that this was an experiment. And in fact, the business of me having a rule in mind was just a ruse. I was really looking for confirmation bias.

Let's say that you have this idea that all orange things fall at the same speed on Fridays. You test this by dropping a carrot and an orange on a Friday morning, and find that the experiment bears out your hypothesis. So you repeat the experiment several Fridays in a row, with every orange object you can find, and always get the same result. You feel pretty secure in believing that orange things fall at the same rate on Fridays.

But of course, this test is incomplete: you should also try throwing some non-orange objects into the mix. You should try running the experiment on some day other than a Friday. If you've always done the experiment in the same lab, you should try running it elsewhere (in case there's a local phenomenon interfering with the results). You should try it in a vacuum, in a plane, and so forth.

In other words, you should specifically try to get a negative result. If you only perform the experiment in a way that you expect will confirm your result, you're giving in to confirmation bias, because you could be wrong and never know it. Think of the scene in Blade Runner where Tyrell says "I want to see a negative before I provide you with a positive" (except, of course, he has his own hypothesis that he expects to see confirmed). Or how Phil Plait tested the notion that you can balance an egg on the equinox by balancing eggs on days other than the equinox.

That's the sort of thing I was looking for in the game. And as you can see, every single time, the players provided an example of the rule they had in mind. No one tried "three different numbers: 4, 4, 4", "a set of integers: 1.8, 2.8, 3.8".

This experiment wasn't my idea. I read about it somewhere, but I don't remember where, so I can't give proper credit. Wherever it was, it said that members of the general public generally came up with examples that matched the rules they were testing, I was curious to see whether a group of self-professed skeptics familiar with the scientific method would behave differently.

It also seems that the players let themselves be railroaded by the examples I provided. No one guessed "a series of numbers: 1, 2, 3, 4", or "a set of three things: rock, potato, Karl Marx".

But I guess the most important lesson to be learned here is that just because you're a skeptic doesn't mean you're immune from poor reasoning. When you read in the scientific literature or on a skeptic's blog about a common mistake that people make, there's a tendency to think "Oh, I'm better than those idiots. I know about this mistake, so I'll avoid it. But avoiding common human mistakes is hard, and takes practice. Here was a room full of people who are interested in science and skepticism, who know perfectly well what confirmation bias is and how it affects people's perceptions, but they still fell into the trap that I laid, because they weren't expecting it.

Placebo, God of Indifference

Today's Speak With Monsters takes a critical look at curing with leeches. (NB: Speak With Monsters makes most sense if you're familiar with the Monster Manual, or at least know about AD&D.)

Cracked on Psychics

Cracked has an article entitled "5 Cheap Magic Tricks Behind Every Psychic". The introductory paragraph reads:

I got into magic at the age of five. I stopped thinking psychics were real at the age of five-and-a-half. Mainly because most of them were doing tricks I had just read in the colorful magic book I had bought for three dollars the week before.

Now go read the whole thing.

Critical Thinking 101

Another excellent video from QualiaSoup:

(HT Attempts at Rational Behavior.)

Homeopathy FAQ

How does homeopathy work?

Lays out everything you need to know.

Full Bigfoot Footage

(Via Pharyngula.)

I Just Had a Supernatural Experience

Spoiler: not really.

I was just in the kitchen fixing dinner for the cats, when I heard a woman cough. Just a single throat-clearing cough. It definitely sounded like a woman.

And this was odd, since there aren't supposed to be any women in the house. Any company would be unexpected, seeing as how I'm snowed in by Snowmageddon. But it was convincing enough that I checked the living room and looked for fresh tracks in the snow outside. Perhaps someone got stranded and wandered in when she found an accidentally-unlocked door? Or could it be a ghost?

Obviously, there was no one there, as you know, since you've read the spoiler at the top. I tried replaying the sound on the tape recorder of my mind^[1] and play it back, trying to figure out what I'd heard, rather than what I thought I'd heard. That is, I asked myself what sounds I might mistake for a woman's cough.

The most likely candidate I came up with is a chair leg creaking across the wooden floor. This is consistent with one of the cats jumping off of the chair to get his dinner, especially since there's a chair that he particularly likes to lie on. I don't know that that's what happened, of course, but it makes a heck of a lot more sense than ghosts. (Besides, $CAT sashayed into the kitchen with his tail held high, and not at all as if he'd seen a ghost.)

At any rate, this illustrates why one should be skeptical about reports of UFOs, the paranormal, etc. When someone says "I saw X", usually what they mean is "I saw something that I interpreted as X". Yes, they're sincere, but it's quite possible for people to be sincerely mistaken. And unfortunately, it's not possible to reach into people's heads and pull out what they actually saw or heard, as opposed to what they say they saw or heard. Which means that oral testimony, however sincere, isn't sufficient to prove the existence of UFOs or paranormal phenoma. As the kids say, "Pics or it didn't happen."

(HT to Anne for the meteorologist video.)

(Cross-posted at Epsilon Clue.)

---

[1] Note to young people: in ancient times, a tape recorder was a device that recorded sounds so you could play them back later. They were very good at playing back ambient sounds. So if you recorded an interview with someone and played it back, you'd hear the creaking of your chair with perfect such clarity that it would drown out the interviewee.

Fishy Invitation

On a social network that I'm on, I recently received an invitation to attend a Landmark Education seminar. The invitation included this description:

A breakthrough is often thought of as a one-time event—a quantum leap that moves us “outside the box.” This seminar addresses how to operate powerfully once we are in that new territory—how to manage and sustain a breakthrough and have that new place be a platform for generating the next level of living.

Breakdowns are an integral and critical part of any breakthrough. When breakdowns are seen as stepping stones to a breakthrough - something that was not predictable becomes possible. Breakdowns occur only against a background commitment - they are an occasion for extraordinary action, for making something happen that would not have happened otherwise.

This set off my BS-O-Meter: vague generalities with positive connotations, and a high buzzword density: "quantum leap", "outside the box", "breakthrough", "platform", "the next level".

The web site's What People Say page has the testimonials (read: anecodtal data) that one might expect, but also a section called "Independent Research", which I thought would be more objective. Unfortunately, none of the links there point to studies published in peer-reviewed publications. If this program were half as effective as its proponents claim, and they could prove it, surely they'd want the academic repute that would come with a published paper. Instead, the "full study" at the top of the list looks more like a sales brochure, or at best a poorly-done newspaper article, than a psychological or sociological study: no abstract, no references, no detailed discussion of methodology. That pushed my BS-O-Meter further into the red.

The people at the JREF forums aren't as kind as Wikipedia or me. My favorite line from that thread is

I went through one of the seminars years ago. A key stepping stone on my path to skepticism.

In the end, while I don't think the organization qualifies as a cult, it does seem to be a cross between life coaching (as seen on Penn & Teller's Bullshit!) and an Amway-style recruiting scheme. I don't think I'll be attending the seminar.

Ray Comfort, plagiarist?

Looks like Ray Comfort found it too hard to write a 50-page introduction to Origin on his own: Metropulse.com, a Knoxville, TN local paper, has a story about Stan Guffey, a University of Tennessee lecturer who wrote a brief bio of Charles Darwin. Turns out that bio bears a striking resemblance to the first few pages of Comfort's introduction (you know, the part that isn't batshit crazy).

(HT Unreasonable Faith and AIG Busted.)

I find it ironic that the approach investigators use to detect plagiarism are similar to that taken by biologists to find homologies, which are one of the bits of evidence pointing to common descent.

So maybe Ray can use creationist arguments in his defense: "You cherry-picked your examples to make your case. If you look at the other 47 pages of the introduction, you'll see that it's nothing like anything Dr. Guffey has written", or "Similarities do not mean that I copied from Guffey. It's more likely that both texts were written by God." Or the ever-popular "Did anyone see copying take place? Then how do you know it happened?"

(Cross-posted at Epsilon Clue.)

Cookies!

The gel electrophoresis cookies I mentioned at the meeting yesterday.

With links to lots more sweet sciencey confectionery.

Lobster signal!

Maybe we should get one of these.

(HT everyone's favorite arboreal crustacean skeptics, the Tree Lobsters.)

Dan Brown, The Secret, and the "What the Bleep" drinking game

Popdose is nominally about pop culture: music, movies, and whatnot. But in a post about Dan Brown's book The Lost Symbol, Jack Feerick had a go at all sorts of woo.

I'll only quote the What the (bleep) Do We Know? drinking game

• Every time someone says something evasive, take a drink.
• Every time someone takes an unjustifiable leap of logic, take ten drinks. Just because.
• Every time Marlee Matlin sighs, take a drink.
• Every time an unsourced anecdote is presented as fact, claim to take a drink.
• Every time the words “QUANTUM PHYSICS!” are uttered, take a drink and do not take a drink, simultaneously.
• Every time someone says “We can’t explain it,” or some variation thereof, take a drink.
• Any time somebody actually explains something… never mind, it won’t happen.
• Any time somebody proposes a violation of Newtonian physics, untake a drink.

(You might want to have a priest and an ambulance handy.)

because you should really go read the whole thing.

Bad Math at the Comcast Center

If you were at the Comcast Center today to see president Obama's speech, you may have caught Secretary of the Interior Gary Locke's opening act introduction.

In it he said that last year, health insurance premiums went up 15%, then asked "How many families' income went up 15% as well to make up for that?"

It's a fine rhetorical question, but the math is wrong: only part of a family's income goes toward paying health insurance premiums, so the total income needs to go up by a smaller percentage to keep everything else equal.

To illustrate with some completely made-up numbers: let's say a family makes $50,000 and spent $10000 of that on health insurance in 2007, leaving $40,000 for everything else. Premiums went up by 15%, or $1500, which means that in 2008, they had to pay $11500.

So in order to have the same $40,000 to spend on everything else, their total income would have to go from $50,000 to $51,500. That's an increase of 3%, not 15%.

Implication

Implication is one of those things we use every day, in "if... then" sentences. In mathematical notation, it's written "p → q", and it's defined as:

if p is…	and q is…	then p → q is.
true	true	true
true	false	false
false	true	true
false	false	true

What trips most people up about it is the last two lines: how can it be true if the premise is false? An example should illustrate why it's defined this way.

Let's say that at the beginning of the semester, the instructor goes over the syllabus and grading, then says "Oh, and if you get 100% on the final, you automatically pass the course." Here, p is "you ace the final" and q is "you pass the course". At the end of the semester, one of four things will happen:

1) You ace the final, and pass the course.
2) You ace the final, but fail the course.
3) You don't ace the final, but still pass the course. This can happen if, say, you got 90% on the midterm and the final.
4) You don't ace the final, and don't pass the course. This can happen if, say, you get 20% on both the miderm and the final.

In the third case, I think we can agree that it would be unreasonable to storm into the instructor's office and say, "Hey! I didn't ace the final, but I passed anyway! You lied at the beginning of the semester!". It would be even more unreasonable, in the fourth case, to storm in and say "Hey! I failed the tests, and you failed me for the course! You lied at the beginning of the semester!".

In fact, the only case in which you'd have a legitimate grievance is the second case: "Hey! I aced the final, but you failed me anyway! You lied at the beginning of the semester!". (Note that I'm using hyperbole. I don't actually advocate yelling at instructors. Gregory House can get away with it because he's portrayed as an indispensable genius. You are most likely not an indespensable genius.)

This also has implications (see what I did there? I slay me) for how we evaluate claims. Again, let's look at a concrete example: let's say that I tell you that I can make it rain by washing my car. That is, I'm making a statement of the form p → q, where p is "I wash my car" and q is "it rains".

You are, of course, dubious. So I rattle off a list of times when I washed my car, and it rained within the following 24 hours, backed up by photos and NOAA archives. What should you do to see whether my claim is true or not?

You might be tempted to look at times when I didn't wash my car, but it rained anyway. In other words, where p ("I wash my car") is false, but q ("it rains") is true. But as we've seen above, "false → true" is itself true, and even if you find a hundred rainstorms when I didn't wash my car, it doesn't mean I'm wrong. Nor would it help to find cases when I didn't wash my car, and it also didn't rain. It does, after all, rain for reasons other than my clean-car fetish.

If you look at the table above, you'll see that there's only one case where implication is false. In fact, "p → q" can be rewritten as "it is not the case that p is true and q is false". This is a cumbersome way to put it, but it provides us with the way to proceed.

What you need to do is come up with a list of cases when I did wash my car, but it didn't rain. If you can come up with such a list, then you can definitely say that I'm wrong.

I think the reason this is confusing is that in colloquial speech, when people say "if" (p → q), what they really mean is "if and only if" (p↔q). When parents say "if you don't eat your carrots, you can't have dessert", they also mean "if you do eat your carrots, you can have dessert". If your friend says "If I'm free on Thursday, I'll meet you for dinner", we tacitly understand that as meaning, "If I'm not free, I won't meet you". But if we want to be careful in separating fact from fiction, we need to understand what claims are being made, and how to test them.

The credulous leading the gullible

The Washington Post has an interesting article about Nigerian email scammers. Near the end, there's this tidbit:

But in these tough times, the scammers said, they are relying more on a crucial tool: voodoo. At times, Banjo said, he has traveled six hours to the forest, where a magician sells scam-boosters. A $300 powder supposedly helps scammers "speak with authority" when demanding payment. A powder, rubbed on the face, reportedly makes victims viewing the scammer through webcams powerless to say no.

"No matter what, they will pay," said Olumide, a college student, adding that he is boosting his romance scams by wearing a magical, live tortoise hanging from a cord around his neck.

It's bad enough if you get scammed, but it's worse if the guy scamming you thinks that wearing a live tortoise around his neck will improve his love life.

Hm... maybe rubbing Testudo's nose has nothing to do with passing exams after all.

Dara O'Brien mocks homeopaths

My favorite line: "I would take homeopaths and put them in a big sack with psychics, astrologers, and priests. And I'd close the top of the sack with string. And I'd hit them all with sticks. And I really wouldn't worry about who got the worst of it."

Via Bad Astronomy.

Interview with Tim Farley

The June 19 episode of American Freethought features an interview with Tim Farley, the guy behind Whatstheharm.net, a site that catalogs the very real costs (in both money and human lives) of lack of critical thinking.