Science That Matters

The experiment was stunning in its simplicity. A group of teachers at a low-income South San Francisco elementary school were asked to begin the year by administering the “Harvard Test of Inflected Acquisition” to their students. The results were processed and the teachers were given back a list of students whose intellectual abilities were expected to “bloom” that year. At the end of the school year the test was administered again and, sure enough, the bloomers were found to have bloomed, surpassing the other students. But there was just one catch: the test was actually a simple IQ test and the “bloomers” were actually chosen randomly.

The result was called the “Pygmalion effect”: teachers who expected their students to do better actually caused their students to do better. It was a classic self-fulfilling prophecy. The study (published as Pygmalion in the Classroom) was widely hailed. It made the front page of the Times, The Today Show, the New Yorker, and Time, among others. Teacher workshops in avoiding the effect spread from Puerto Rico to Saudi Arabia. LA banned IQ tests in its elementary schools. Presidents, textbooks, and Wikipedia articles repeat the notions to this day, over 30 years later.

Except none of it was true. The original study was conducted in first through sixth grades. The results were only statistically significant in grades one and two (where the alleged bloomers started with a 4-point advantage). The study was repeated in two Midwestern schools, where as statistically significant advantage was found in favor of the kids who weren’t expected to bloom. Psychologists who reviewed the analysis of the IQ test results found something was badly wrong. Some kids got lower IQ scores than they would have had they just filled out the test randomly. (It turned out the kids just didn’t fill out the test.)

All of this data was available before the Pygmalion book was ever published or promoted. Yet it was glossed over or otherwise ignored by the authors. And even when critics published articles spelling out the details, their critiques have been largely ignored by the public. Harvard’s Robert Rosenthal, the author of the original study, tried four more times to reproduce the effect, failing each time. A handful of studies did reproduce the effect, but they had incredibly small sample sizes. A meta-analysis found no overall effect when sample size was taken into account and showed that nearly half the replications had results that went in the opposite direction.

It might be nice if this Pygmalion effect were real, if students could do better on IQ tests simply by having their teachers think more highly of them. But, as best as we can tell, wishing doesn’t make it so.

• Samuel Wineburg’s critique
• Robert Rosenthal’s response
• Wineburg’s rebuttal

In 1972, Dr. Myron L. Fox, an authority on the application of mathematics to human behavior, gave a lecture to a group of educators — psychiatrists, psychologists, social workers, education students, and administrators — on the topic of “Mathematical Game Theory as Applied to Physician Education.” He spoke for an hour and took another half hour of questions. According to feedback forms distributed after the lecture, the talk was very well received. “Excellent presentation, enjoyed listening,” commented one. “Has warm manner,” added another. “Good flow, seems enthusiastic.” Not everyone was so positive, though. “Too intellectual a presentation,” complained one. “My orientation is more pragmatic.” Still, the majority of the responses were broadly favorable.

There was, however, a more serious problem. “Dr. Myron L. Fox” was actually an actor trained to give a speech consisting largely of “double talk, neologisms, non sequiturs, and contradictory statements … interspersed with parenthetical humor and meaningless references to unrelated topics.” The speech was actually an experiment conducted by a group of professors of medical education. They summarized the results by noting that “no respondents saw through the hoax of the lecture, [] all respondents had significantly more favorable than unfavorable responses, and [] one even believed he read Dr. Fox’s publications.”

“Given a sufficiently impressive lecture paradigm,” they concluded, “an experienced group of educators participating in a new learning situation can feel satisfied that they have learned despite irrelevant, conflicting, and meaningless content conveyed by the lecturer.”

• Read the study

There has been a fair amount of media buzz about the parasitic protozoan toxoplasma gondii. It’s a parasite whose primary lifecycle involves transmission between cats (who eat infected rats) and rats (that eat fæces from infected cats). The strange thing about this parasitic disease, which is called toxoplasmosis, is that it changes the behaviour of the rats, slowing their reaction times and making them less fearful — of things like cats.

It’s a neat trick for the parasite. But one wouldn’t want brain-altering protozoa to get into the wrong place!

Rats and cats are not the only homes for toxoplasma gondii. Human beings can also contract toxoplasmosis, and indeed it is endemic around the world. It can be contracted either by eating undercooked meat or by ingesting trace quantities of cat fæces (yuck, but it happens). The prevalence of the latent form of the infection ranges widely — 4.3% in South Korea; 22% in Britain and the U.S.; 80% in France. These rates (and the fact that the high risk window for transmission by cats is only a couple of weeks) suggest very strongly that meat is the primary vector for human infection. Toxoplasmosis rarely causes obvious symptoms, except in people who are immuno-suppressed or pregnant, where acute infection can occur and is extremely serious. This is why pregnant women are told to aovid cats and undercooked meat.

Latent toxoplasmosis rarely causes obvious symptoms. But what might those tricks aimed at altering rat brains do to us? A rash of papers on the subject have appeared in the last few years, many of them produced by a single research group in the Czech Republic. These results correlate toxoplasmosis with — and sometimes accuse the parasite of causing — the most extraordinary things: making women more promiscuous; making them bear more male children; making men less intelligent and more jealous and anti-social; causing schizophrenia; slowing reaction times. Some of the results about personality are a little tenuous. But one of the most striking results was not at all tenuous: a 2002 study published in BMC Infectious Diseases finding that toxoplasmosis infection rates are very high among drivers who have been involved in car accidents, around three times those in the general population. This study was originally conducted in the Czech Republic, but was succesfully replicated in Turkey [PDF]. The correlation is real.

It’s difficult to believe that an obscure parasite could be causing such sin and misery, and we still don’t know that it is. The question is whether there are any plausible causes that could be common to both toxoplasmosis and the slow reaction times (or something else) that result in car accidents. The most obvious variable one would reach for is social class. Often, bad things — like diseases and car accidents — are more likely to happen to the poor. But toxoplasmosis may be an exception. Perhaps wealthy people eat more French food, and are more likely to drive in countries like the Czech Republic and Turkey (we’d like to see this tested). Another variable that has been suggested is “risk taking behaviour”, but it seems incredible that risk takers would really eat that much more rare meat.

We here at Science That Matters have another variable we propose testing: immune system strength. A weak immune system could lead to both latent toxoplasmosis and to other symptomatic conditions that increase the risk of car accidents. If some data becomes available to refute that possibility… we will be persuaded that parasites are in fact a major cause of car accidents.

Rats were given a choice. They could take a test where, if they succeeded, they were given a large reward and if they failed no reward. Or they could decline the test and receive a guaranteed small reward. When the test was easy, rats took it and got the big reward. But when it was hard, they declined. And, indeed, when they were not given the option of declining, they did poorly on the hard test. So rats accurately know when they don’t know something. Someone page Donald Rumsfeld.

• The study (sub. req.)
• Write-up in AAAS’s ScienceNOW
• The study: Metacognition in the Rat [PDF]

The China Price Project tried to figure out what causes China’s low, low prices, the prices that have allowed it to capture “70% of the world’s market share for DVDs and toys, more than half for bikes, cameras, shoes, and telephones; and more than a third for air conditioners, color TVs, computer monitors, luggage, and microwave ovens.”

Their conclusions are that lower labor costs account for 39%, China’s clever practice of locating related factories near each other (which lowers shipping costs and makes it easier to coordinate) accounts for 16%, and the efficiencies permitted by the flood of investment into China account for 3%. As for more standard government policy issues, export subsidies account for 17%, undervalued currency for 11%, counterfeiting and piracy for 9%, lax environmental and safety standards for 5%.

• Report of the China Price Project

The Microsoft Typography group (one of the few good groups at Microsoft, it seems) gave subjects two versions of a piece of text, one with careful typography, one with crummy stuff. Which version they had didn’t matter for reading speed or comprehension and subjects didn’t even tend to prefer one. But when you looked at measures of happiness — like activation of the smiling muscles or better performance on simple tasks — the half that got the good typography were much happier.

• A description of the study (from UIScape, a sort of Science That Matters for CHI research)
• The Aesthetics of Reading [PDF paper]
• Proceedings containing the full paper

The Bible, as any casual reader of The Brick Testament knows, is an exceedingly violent book. And any reader of the news knows that violent people often use God to justify their actions. But is this just coincidence?

Brad J. Bushman, a psychologist who studies the effects of TV and video game violence, recently joined Robert D. Ridge, Enny Das, Colin W. Key, and Gregory M. Busath in testing the question. They took some Godful students at Brigham Young University and some atheist students at a university in Amsterdam and split them into two groups. Half heard a story about how an Israelite couple visited the town of Gibeah, where the townspeople raped and beat the woman to death. The Israelites retaliated by attacking the town. The other half heard the same story, but with an additional paragraph noting that God endorsed the retaliation.

Then the students were placed in a competitive game where they could blast noises into the ears of their opponents of variable volume. For students who believed in God, those who heard the passage in which God endorsed the retaliation made the noises as loud as possible twice as often as those who only heard the violent story. For students who did not believe in God, the increase was 40%.

• Read the paper
• Read the write-up in Nature
• More Brad Bushman studies

I hesitated before posting this piece. Is computer science education really a subject in which any result could be of tremendous sociopolitical importance? But, upon consideration, the answer has to be “yes”.

The ability to understand how computer programs work is arguably a central part of a well-rounded modern education, in the same way that people once needed to know a bit about horses. Logically, as a friend of mine once argued, programming should probably be taught sooner than calculus, because it develops the same kinds of analytic reasoning skills but is useful to more people. Perhaps one of the reasons that programming isn’t more widely taught is that (apparently) many students have terrible trouble learning it. Others find it utterly straightforward.

A remarkable little working paper by Saeed Dehnadi and Richard Bornat provides intriguing data which illuminates the underlying cognitive step that is taken by those who can learn to program and missed by those who cannot. It turns out that while this mode of thinking is loosely correlated with general educational achievement, the link is not determinate. Some high achievers cannot program, and some low achievers can. Most remarkably, there are other kinds of abstract analytical tasks where performance is very well predicted by educational achievement. Programming just isn’t one of them.

The article is tentatively titled The Camel Has Two Humps. Be sure to get to the last set of graphs.

It appears that these results should open up an entire new field of research on testing the authors’ proposed explanation, polishing the tests that predict ability to learn programming, and — most importantly — figuring out ways to teach programmatic rule structures to those who do not understand them intuitively.

A fascinating survey was conducted by Time magazine for the 2000 U.S. election cycle (disected later in an op-ed piece by David Brooks in the New York Times). It’s rare that a few numbers say so much about a nation’s misunderstanding of itself.

When Americans were asked if they are in the wealtheist one percent of the population, 19% of them said “yes”. A further 20% of them said “no, but I will be at some point in my life”.

America’s foundational story is that anyone, however lowly, can get rich. This is the land of opportunity. But, according to the data, a poor child’s odds of striking it rich are better in Europe.

Thousands of studies have been done claiming that addiction is a disease, mostly by putting rats in a cage with some drugs and noting that they’ll repeatedly take the drugs, even if it means starving to death.

Bruce Alexander was skeptical about these results. He noticed that the rats in the experiments were stuffed alone in a boring cage with little else to do. “If I was strapped down alone in a cage,” he thought, “I’d probably want to get high too.”

So he built a rat park — a large, intricate, brightly-painted and heavily-padded structure to make the rats actually happy. He put half the rats in the normal cages and half in the park and gave both equal access to drugs.

The rats in the cage got addicted, while the rats in the park stayed away.

Then, even more strikingly, he took rats who’d had 57 days to get addicted to the drugs and took half of them out of the cages and put them in the park. The rats, even though they’d been addicted in the cage, suddenly stayed away from the drugs. They even voluntarily detoxed — trembling and shaking, but still staying off the drugs.

The top-shelf journals like Science and Nature rejected the study. It did end up getting published in a peer reviewed journal (Pharamacology, Biochemistry and Behavior, impact factor 1.5), but received little public attention. His university pulled the funding for the project.

• The paper [PDF]
• Detailed Wikipedia article
• Related paper by the study’s author