Who truly governs America’s cities?

Filed under Disproof, Politics, United States by aaronsw on April 24 | 2 comments

Who Governs? is a widely-hailed classic in the field of political science; it was the book that basically made the career of “the Dean of American political scientists”: Robert A. Dahl. In it, Dahl attempts to discover how government really works in America. To do this, he decides to study decision-making in a typical American city — namely, the one outside his office at Yale University: New Haven, Connecticut.

Who Governs? argues that New Haven worked according to Dahl’s theory of “pluralism”: elite political groups exist, but they aren’t very powerful. Instead, they balance each other out, leaving politicians (and thus their voters) firmly in control.

Fifteen years later, the political sociologist G. William Domhoff went over the issues covered in the book (including Dahl’s notes and sources, which Dahl was honest enough to share) only to find that Dahl had badly bungled the research. Upon closer review, Dahl’s own notes, plus a few new sources, revealed exactly the opposite story. This is the story of how Dahl got things so badly wrong.

Finding the elite

Dahl begins by claiming that there’s little overlap between the city’s social and economic elite, part of his argument that different groups of elites balance each other out. So he counts company presidents, individuals with significant property in the city, directors of multiple sizable city firms, and any director of a bank in the city. Then he takes this list and sees how many of them attended the New Haven Lawn Club debutante ball. He doesn’t find much overlap.

Domhoff points out that this is kind of an odd metric. For one thing, not all the elites go to the debutante ball, while many people from out of town do. So instead he says any member of one of New Haven’s three elite social clubs is a social elite, while anyone who’s a director of one of New Haven’s ten most interlocked firms is an economic elite. (Firms are interlocked when they share members of their board of directors.) He finds incredible overlap — of the entire corporate network, 55% are in a social club; of those on two boards, 80% are. So much for that.

Deciding urban renewal

But the bulk of Dahl’s study is his attempt to see who actually makes decisions on three important issues. He picks (arbitrarily) political nominations, public education, and urban renewal. For each, he interviews the major players to find out how the relevant decisions got made. Domhoff points out that political nominations are rather uninteresting, since they’re internal party disputes, and that elites don’t care about public education, since they all live in the suburbs or send their kids to private schools. Which leaves urban renewal.

New Haven went through a massive urban renewal shortly before Dahl’s study and Dahl claims it was orchestrated by the city’s mayor, who heroically fought resistance-to-change on all fronts, selflessly ensuring what was best for New Haven. (The urban renewal project in fact ended up completely destroying New Haven’s downtown, but that’s a separate story.) As Dahl quotes the mayor: “Redevelopment in New Haven began in February of ’55. We had to start from scratch and assemble a team and start to file all the papers and get the whole program launched.” But Dahh omits a key piece of context.

Urban renewal had in fact been in the works for years, at the insistence of the town’s Chamber of Commerce. When the new mayor took office, the Chamber of Commerce quickly organized a meeting with him at which “the entire program [of urban renewal] would be explained to him and he would be urged to get action started on the program” (as their own minutes described it). A representative met with Mr. Lee at one of the elite social clubs and reported back that “Mr. Lee said he was in entire agreement with [our] program for action.”

So why did Lee claim that he had to start from scratch? Turns out, the city was having trouble getting some of their filings approved, so they decided to try a new strategy and assemble a new team, which begun by refiling all the relevant permits. But this was just a technical detail — the urban renewal plans themselves had long been in the works.

Normally in science, you refute someone’s results by conducting the same form of research yourself under different circumstances. But Domhoff went much further: he reexamined the very same research that Dahl conducted, even using Dahl’s own notes and transcripts. But the conclusions he came to were wildly different. It’s hard to think of a more stunning refutation. Not that political science was interested in hearing it. Dahl remains the field’s idol, while Domhoff is an obscure professor at UC Santa Cruz.

Evo psych error roundup

Filed under Disproof, Psychology by aaronsw on April 24 | 5 comments

An influential group of biologists, psychologists, and other busybodies has for decades promoted the idea that the social sciences should be grounded in the ideas of evolution, that human behavior should be predicted from estimates of what evolution would do. The idea has been heavily promoted from the 1970s, when it was called sociobiology, until today, where it’s called evolutionary psychology (evo psych for short), but little in the way of compelling evidence has been produced. Today, we’ll focus on some less than compelling evidence.

Exhibit A: One common (and characteristically offensive) claim among evopsychers is that your mother’s mother will spend more time caring for you than your father’s mother because — naturally enough — your father’s mother isn’t evolutionarily certain that you have her DNA, since your mother could have been impregnated by any one of tons of guys. The data does indeed seem to bear this out, but sadly this is no win for the evopsychers, since there are some perfectly competent alternative explanations: kids are usually primarily raised by their mothers and its not surprising that those mothers will look to their mothers for help. (via Jeremy Freese)

Exhibit B: In 1995, Christenfeld and Hill argued that since fathers were so unsure if kids were really theirs, evolution would ensure that kids looked more like their fathers than their mothers, so that they wouldn’t be abandoned by deadbeat dads. And, sure enough, they had some students rate whether kids looked more like their father or mother and found that they looked more like their father. Robert French later redid the study, only to find that he couldn’t replicate the results. Oops. (via Mark-Jason Dominus)

Exhibit C: In 1993, Devendra Singh spent months pouring over old copies of Playboy — for science, of course. He set about measuring the waist-to-hip ratios of Playboy models and Miss America winners, concluding that they had maintained relatively constant — approximately .70 — even as the models had gotten thinner over the years. He argued that men were evolutionarily wired to find this “hourglass shape” attractive. The result was quoted in just about every evopsych textbook and news article since. Well, Jeremy Freese and Sheri Meland checked the numbers and found — once again — that none of it was true. There have actually been statistically significant changes in waist-to-hip ratios over time. (original article)

The power of hope

Filed under Disproof, Education, Psychology by aaronsw on April 22 | 0 comments

The experiment was stunning in its simplicity. A group of teachers at a low-income South San Francisco elementary school were asked to begin the year by administering the “Harvard Test of Inflected Acquisition” to their students. The results were processed and the teachers were given back a list of students whose intellectual abilities were expected to “bloom” that year. At the end of the school year the test was administered again and, sure enough, the bloomers were found to have bloomed, surpassing the other students. But there was just one catch: the test was actually a simple IQ test and the “bloomers” were actually chosen randomly.

The result was called the “Pygmalion effect”: teachers who expected their students to do better actually caused their students to do better. It was a classic self-fulfilling prophecy. The study (published as Pygmalion in the Classroom) was widely hailed. It made the front page of the Times, The Today Show, the New Yorker, and Time, among others. Teacher workshops in avoiding the effect spread from Puerto Rico to Saudi Arabia. LA banned IQ tests in its elementary schools. Presidents, textbooks, and Wikipedia articles repeat the notions to this day, over 30 years later.

Except none of it was true. The original study was conducted in first through sixth grades. The results were only statistically significant in grades one and two (where the alleged bloomers started with a 4-point advantage). The study was repeated in two Midwestern schools, where as statistically significant advantage was found in favor of the kids who weren’t expected to bloom. Psychologists who reviewed the analysis of the IQ test results found something was badly wrong. Some kids got lower IQ scores than they would have had they just filled out the test randomly. (It turned out the kids just didn’t fill out the test.)

All of this data was available before the Pygmalion book was ever published or promoted. Yet it was glossed over or otherwise ignored by the authors. And even when critics published articles spelling out the details, their critiques have been largely ignored by the public. Harvard’s Robert Rosenthal, the author of the original study, tried four more times to reproduce the effect, failing each time. A handful of studies did reproduce the effect, but they had incredibly small sample sizes. A meta-analysis found no overall effect when sample size was taken into account and showed that nearly half the replications had results that went in the opposite direction.

It might be nice if this Pygmalion effect were real, if students could do better on IQ tests simply by having their teachers think more highly of them. But, as best as we can tell, wishing doesn’t make it so.