Reporter Degrees Of Freedom


A sample of Thursday’s talk at Yale:

These are four headlines describing the same study, Milkie, Nomaguchi and Denny (2015). The study found that of twenty or so outcomes, only three of them – all measuring delinquent behavior among teenagers – show significant effect from time spent with parents (and this result remains after Bonferroni correction). So Vox has a great argument for their headline. The National Post has an okay argument for their headline even though it’s kind of cherry-picked. The Washington Post just sort of reads between the lines and figures that if it’s not quantity of time that helps kids, it must be quality. And FOX also reads between the lines and figures that if moms spending time with their kids has no effect, the argument from opportunity costs suggests mothers are spending too much time with their kids.

None of them are completely outright lying. And indeed, most of the articles eventually explain what I just said, halfway down the article, in one or two short sentences that most readers will skim over. But the rest of the article uses the study to support whatever the news source involved wants it to support, and so people will come up with four diametrically opposed conclusions from this one study depending on which source they read.


Here’s a study that I wasn’t able to include in the presentation because it just came out recently. As per the Rice University press release: Overweight Men Just As Likely As Overweight Women To Face Discrimination.

The paper included two studies. In the first, men went into stores either with or without fat suits and try to do some things – ask if there were job openings, ask for a job application, ask an employee for help, try to buy some things, et cetera. Then they measured the men’s success across both conditions to see if they had more trouble when they appeared overweight.

In the second, subjects were asked to rate videos of an employee giving a marketing spiel for a new product; once again, the employee was either wearing or not wearing a fat suit. They measured the subjects’ ratings across conditions to see if they ranked the overweight employees lower.

The first study only included men, and so could not possibly have determined whether men were more or less likely than overweight women to face discrimination. The second study actually did have both male and female employees involved, and although it really wasn’t their main interest, the researchers did a post hoc evaluation to find the effect in each sex. In all three of the outcomes where discrimination was found, women faced more discrimination than men. They didn’t significance-test the comparison, but just from eyeballing it, it was probably significant.

So a paper in which one study does not compare men to women, and the other study finds women facing more discrimination than men, the press release somehow gets phrased as “Overweight men just as likely as overweight women to face discrimination in retail settings”. Huh.

You might wonder, “Does it really matter what a press release says? Does anyone read the exact wording?” Yes. Many other news sources copied the phrasing, for example Medical Daily’s Fat Discrimination Is The Same Regardless Of Gender. One such copycat, copied the press release nearly word for word, including the title. Then it got posted on Reddit and now has 5189 upvotes and 1572 comments. So there’s that.


“But at least it correctly raised awareness of how weight discrimination is a big problem in the retail setting, right?”

The paper measured a ton of different outcomes. Let’s focus on Study 1. The actor in the fat suit was supposed to ask if there were job openings (there were) and see if the company told him. Then he was supposed to ask for an application form and see if they gave it to him. Then he was supposed to walk in as a customer and see if employees greeted him. Then he was supposed to ask the employees to recommend him an item and see if they did. Then he was supposed to ask them to recommend him a second item and see if they did.

No difference was found between overweight and normal-weight actors in any of those five experiments. Two of them had ceiling effects that probably made the attempt futile, but the other three didn’t, and there wasn’t even a trend toward discriminating against the overweight guy.

So what did they find discrimination on? They say that detected “interpersonal discrimination”, ie discrimination based not on any quantifiable outcome but based on how friendly/warm the person interacting with the actor seemed toward him. They determined this by self-rating and other-rating; that is, the actor wrote down how friendly he thought the store clerks were toward him, and a spy surreptitious observer who had placed herself near the interaction also rated this for corroboration. Their rating scales included twelve items including “how many times did the clerk nod”, “how friendly did the clerk seem?”, “how much eye contact was the clerk making?” and “how much comfort level did the clerk seem to have?”. The experiment found a statistically significant difference between the fat-suit-wearing and non-fat-suit-wearing trials and concluded that there was interpersonal discrimination.

But hold on a second! The study says nothing about anyone being blinded. In fact, it’s really hard to blind an actor to the fact that he is going into some stores while wearing a fat suit and other stores while not wearing a fat suit. As far as I can tell, everybody involved was in on the study from the beginning. If your boss tells you “I want you to rate how much comfort level clerks have with you for this study on fat discrimination”, it seems really possible to me that there might be a slight tendency to overrate the clerks who interacted with thin-you, and to underrate the clerks who interacted with fat-you.

How slight a tendency? Clerks dealing with fat people got an average rating of 2.3 (seven point scale, lower is better), and those dealing with thin people of 2.0.

So after finding no discrimination on five objectively measurable outcomes, they find very subtle discrimination on an unblinded subjective outcome practically designed to produce placebo effects.

We move on to the second study, where participants (as usual, psychology students) are rating video presentations given by fat vs. thin people. This is supposedly tying into the “retail industry” theme of the paper, but honestly it seems kind of forced to me.

Anyway, the participants are asked to rate their presenters on seven measures: overall quality of presentation, overall attitude toward product being presented, overall attitude toward the store that would employ a person such as this, intention to support the store, employee’s appearance, employee’s carelessness, and employee’s professionalism. The results:

There was no difference between how participants rated overweight vs. normal-weight presentations overall.

There was no difference between how participants rated products presented by overweight vs. normal-weight people.

There was no difference between how participants rated stores staffed by overweight vs. normal-weight people.

There was no difference between how likely participants were to support stores staffed by overweight vs. normal-weight people.

There was a difference in how participants ranked the appearance, carelessness, and professionalism of overweight vs. normal-weight people.

The first four results are encouraging. What about the last three?

Well, I feel like if you ask people to rank someone based on “their appearance”, and your subjects answer based on how they look, you kind of walked into that one. Oh no, people rank conventionally attractive people as having better appearances than less conventionally attractive people! Someone call John Ioannidis to double-check this astonishing result!

“Carelessness” and “professionalism” are perhaps less excusable, but c’mon, you had them watch a two-minute video. When you give someone zero information on a thing, and you force them to make a judgment on the thing, then yes, stereotypes are their best source of information. If you showed me a picture of an average-looking man and an average-looking woman and say “Quick! Which of these people is more likely to like baking cupcakes?!” I’ll pick the woman, not because I think all women are obsessed with cupcakes or because I go around looking at every woman I see as a cupcake factory, but because you asked me a stupid question and ensured stereotypes were the only thing I had to go on.

Then the authors find that this was mediated by explicitly-expressed stereotypes against fat people, which is kind of interesting, but doesn’t make the nonsignificant things any more significant.

So to sum up: there was no discrimination against the overweight on any objective measure of the actual retail experience, including positions advertised, applications given, greetings offered, or customers served. There was also no discrimination against the overweight on presentation evaluations in terms of overall evaluation, evaluation of employee, evaluation of product, or evaluation of company.

There was a tiny amount of discrimination on a subjective measure rated by unblinded observers aware of the purpose of the study. There was also some evidence on three subtler ratings of the presentation that seemed designed to ask participants impossible questions in order to force them to stereotype. However, these meaningless scales did not effect the raters’ overall impressions as measured any of four different ways.


Here is the reporting from the news outlets that passed their first test and didn’t frame it as men and women facing equal amounts of weight discrimination.

Business Insider: Researchers Had Men Pretend To Be Obese – And The Results Are Disturbing, which says that “this research highlights the importance of including men in discussions about weight stigmatization,” and “the authors also advocate organizational efforts to combat negativity against heavy customers and potential employees…the first step may be for individuals to become aware of how strong weight biases are.”

AskMen: Young Men Who Appear Overweight Suffer Interpersonal Discrimination. “Researchers disguised six thin young men as obese customers or job applications and found that they were victims of microaggressions. Basically people were a little bit more jerky towards them.”

Oximity: Overweight Men Often Snubbed At The Mall. “Shopping malls can be hostile places for overweight men, regardless of whether they’re customers or simply looking for a job.”

The Health Site: Men, Here’s One More Reason For You To Lose Weight. “Ruggs said that these findings were another reminder that there was still more work to be done in terms of creating equitable workplaces for all employees, potential employees and consumers. She concluded that this was something organisations could take an active role in, and said that companies could do better job training on customer relations as part of the employees’ new-hire process. ”

Now I’m almost missing the kind of random scattershot media bias we found on the time-spent-with-children study. Here every media outlet reports the results the same way that the study’s author and the press release reports the results.

This is not a totally wrong interpretation, any more than “six hours a week will tame your teen” is a totally wrong interpretation of the childhood study. But if I myself were writing an article on this study, it would be SURPRISINGLY LITTLE DISCRIMINATION FOUND AGAINST OVERWEIGHT MEN, and mention somewhere in the middle that some discrimination was found on a few sketchy variables. Instead, we get DAILY DISCRIMINATION AGAINST OVERWEIGHT MEN and WE NEED TO INSTITUTE SENSITIVITY TRAINING PROGRAMS IN RETAIL ENVIRONMENT, and they mention somewhere in the middle that a lot of important variables came out negative.

A lot of studies work like this. You test ten or twenty complicated variables, you get positive results on some, negative results on others, some of those results seem plausible, other results seem like maybe you made a mistake somewhere or didn’t have enough power or whatever, and then you make an interpretation based on your personal bias. Then it goes from the researcher’s personal bias to the abstract to the press release to the headlines to the mind of the average reader, dropping subtlety at each step, until “No discrimination against overweight men, except where the study was practically designed to ensure false positives” becomes “Rampant discrimination against overweight men everywhere” becomes “Overweight men are discriminated against just as much as overweight women.”

Don’t get me wrong. I expect there probably is lots of discrimination against overweight men. And I think this study’s project of trying to find it and convince people of its existence was worthwhile. But I don’t think you should get to convince everyone that science has proven X, unless science has actually proven X. The process that produced these headlines is strong enough to produce any headline you want, with the part where you actually do the study becoming more and more of a ritual or a formality. There are just too many degrees of freedom between the study and the reporting.

Stalin once said that “those who vote decide nothing; those who count the votes decide everything.” It’s starting to look like those who do the studies decide nothing and those who report the studies decide everything. The only solution is to actually read the study and not just the headlines. Sometimes we might even have to – God help us – read beyond the abstract.

This entry was posted in Uncategorized. Bookmark the permalink.

358 Responses to Reporter Degrees Of Freedom

  1. ??? says:

    “Sometimes we might even have to – God help us – read beyond the abstract.”

    Considering the abstract is usually the only free material available, we have to make do.

  2. Asher says:

    Did they control for heritability?

  3. DensityDuck says:

    The point of the fat-dude study is not “do fat dudes have a harder time than not-fat dudes”.

    The point of the fat-dude study is to allow SFWs to say “Hey MEN, looks like discrimination happens to MEN TOO, so when we say that you should make discrimination be a criminal felony you should TOTALLY BE ON BOARD.”

    • DrBeat says:

      Dude I’m as opposed to the Social Justice Movement as much as anyone but this isn’t even a little bit true.

  4. Loki says:

    Did nobody suggest that the retail employees’ reactions could have been due to:
    * uncanny valley effect of something being slightly ‘off’ about the person’s appearance, namely, they looked like a guy in a fatsuit not an actual overweight person
    * they could tell the guy was wearing a fatsuit and were looking around for the hidden cameras?

    Fatsuits are *not* convincing close-up. Minimum, you’ll notice that the guy is wearing heavy makeup to conceal the joins between his skin and the fake double chin.

  5. Patrick says:

    My proposed headline: “People primed to expect discrimination tend to feel discriminated against even when objective data demonstrates no discrimination occurred. Implications for society as a whole are enormous.”

    • Cauê says:

      Now that’s a study I would like to see. Does it exist?

      • Patrick says:

        Well, it seems like a pretty simple explanation for how the study described above yielded no measurable discrimination except for the self reported impression of actors who knew what they were testing for.

        Good luck getting research money to study that directly, though.

      • Loquat says:

        Sounds like it’s strongly related to Stereotype Threat, where test-takers primed to think that their group does worse on tests do in fact perform worse on tests, compared to members of the same group who weren’t primed.

        • Cauê says:

          I think it looks very different, as this is about perceptions of other people’s behavior.

          This one looks like a quite simple effect, really. Confirmation bias alone would do it.

    • DensityDuck says:

      haw. You could easily turn that around. “People primed to feel discriminated against and self-reporting their feelings only felt a tiny bit discriminated against. That means that when someone feels genuinely discriminated against, there must be ENORMOUS discrimination against them!”

      Oooh, or this: “White men, when primed to feel discriminated against and self-reporting their feelings, still only felt a tiny bit of discrimination. This proves that white men are incapable of identifying discrimination, even when it’s happening to them!

  6. Deiseach says:

    So really all we can conclude from this study is “People who dress up in fat suits feel that shop assistants are not as polite to them” 🙂

  7. And this is why I am a climate skeptic, anti-AGW alarmist. The actual science issue is a vague debate about CO2 sensitivity and feedback factors that nobody is really sure about. We know the 1980’s computers models failed to predict 2015, they needed tweaking, now they really promise it is better now, but nobody is exactly sure about anything there. I support ongoing research, more empirics, less modelling, and an intelligent discussion about the climate. The Al Gore type stuff, which I am opposed to, is just the reporters and everybody else who believes the reporters.

    Some people wonder why this climate skepticism stuff is so closely linked with right-wing politics. The closest answer is that climate change is so much more reporting than science, and there a certain political conflict with reporters and the general direction they tend to distort things towards.

    • James Picone says:


      Why do you think your favourite denialist blog is any better about this than the papers?

      For example, you appear to think that there’s serious debate about sensitivity in the literature. But there are close to no papers calculating a sensitivity lower than 1.5c (and the ones that are there are published in Energy & Environment, which is… not exactly a credible journal), and there’s only a small set of studies showing sensitivity <2c (mostly energy-balance papers). Meanwhile, pretty much everybody posting in the literature is aware that the sensitivity distribution must have a fat right tail because of the fundamental physics involved.

      Similarly, you think that the 1980s models don't match very well – when they certainly got a lot more right than a naive no-change model, and appear to have pretty good feedback value (their /forcing/ values are wrong, partially because back then the consensus CO2 forcing value was about 10% too high).

      I'm totally in favour of there being more research, but I think there's more than enough evidence that we should start reducing CO2 emissions ASAP, as well.

      (P.S. you claim above that government grants should be biased in the direction of "it's a problem" as much as grants from Shell would bias in a "not a problem" direction. If governments are so in favour of GW being a problem, why is it so fucking hard to get them to do anything about it? Meanwhile, we have historical examples of industry muddying the scientific waters in the form of the tobacco industry, CFCs, and the tobacco industry again for second-hand smoke.)

      • Anonymous says:

        AGW, while probably being real, and possibly being caused significantly by human activity, is probably not dangerous. Certainly not as dangerous as people whose grants depend on it being dangerous tell you.

        • James Picone says:

          Oh well I’m convinced.

          Meanwhile, the consensus scientific position is that with 95% confidence >50% of the warming over the last umpteen years is anthropogenic, with the mean estimate for how much warming is anthropogenic being >100%. That scores a ‘possibly being caused significantly by human activity’ from you. Says a lot.

          • Anonymous says:

            Suit yourself.

            My objections are that:
            a) that truth does not arise from consensus,
            b) the parties investigating the issue are not neutral, being subsidized in direct proportion of how much of a problem they report the issue being, especially if the solution to the problem lies within the hands of the parties subsidizing,
            c) one of the two primary undesirable effects, rising sea levels, are manageable even with medieval earthworks technology,
            d) the rising temperatures due to anthropogenic activities are self-limiting, in that they will reduce the very anthropogenic activities that cause them.

          • Nathan says:

            @ James

            Notice that you didn’t actually address the main argument of the comment you are replying to. Whether climate change is human caused is a separate question to whether it is a bad thing and if so how bad.

            Surely you would have to acknowledge at least that the IPCC WGII reports are significantly less rigorous and reliable than WGI (e.g. A much higher reliance on grey literature and on work from unambiguously biased sources like WWF). I don’t think that we can say with any confidence that we have a clear understanding of the net costs/benefits that climate change would incur.

          • James Picone says:

            a) The opinion of experts is a good guide to truth in their area of expertise,
            b) Bullshit,
            c) Maybe if you’ve got a couple of centuries and a lot of money to throw at it; and there are at least three obvious undesirable effects (you’re forgetting ocean acidification I suspect),
            d) Sure would be great if we could avoid significant impacts to economic growth by taking some steps now, instead of waiting to hit the brick wall.

            What argument?

            And sure, the impacts work is not nearly as good as the physical basis stuff. There’s a wide range of possible impacts. But bad outcomes, and really really bad outcomes, are just as much in the range of possibility as little-impact outcomes (beneficial outcomes are not very plausible). I would expect people posting here to understand risk mitigation.

          • Anonymous says:

            @James Picone
            a) Sure.
            b) Nope.
            c) And the policies advocated by AGW alarmists don’t require heaps of money? Do they not require time and effort?
            d) Sure would be great if the solutions themselves weren’t just as limiting to growth as the potential problem.

            Reducing emissions is a good in itself, due to air quality considerations, but is not genuinely proven to do anything much about global warming.

          • @Anonymous

            I’d like to learn more about d)

            I have something sort of a similar opinion. Looking at distressed polar bears is emotionally tough but the planet has survived warmer periods. The biosphere will adapt to it, it can. It is not a huge problem for evolution. It will be different but evolution never ever promised us to keep things the same. Maybe polar bears go the way of sabre-tooth tigers, sad thing, but that is evolution. Perhaps new kinds of cute animals will evolve. So I would ignore the environmental/natural/species impact. I think on the animals and plants level, we have bigger problems, such as overfishing.

            As for humans, vast amounts of land in Canada and Siberia will be suitable for agriculture. This isn’t a bad thing. Everywhere else, more CO2 means more produce. The economic-human impacts are complicated. One thing to consider, however, it will hit countries like Burundi hard, who are already in a bad shape, and it will help countries like Canada, Russia, Argentine, who are in a good shape. I am not advocating some kind of a global redistribution, nor immigration, but I must admit the impact will be disparate. Quite frankly? I propose to help them by providing sterilization.

          • James Picone says:

            A carbon tax is not in the same ballpark of cost or effort as attempting to dike every major coastal city and floodplain in the world for the next several centuries.

            You’re advocating RCP6.0 or RCP8.5 outcomes, depending on how plausible RCP8.5 actually is. I’m not nearly as sanguine as you about the possibility of hitting 2.6c above preindustrial by 2050. I guess you think that humans can adapt to anything – except increases in the cost of fossil fuels, of course.

            Rates of temperature change this rapid are not very common in the paleontological record. Most of the ones that do occur are associated with supervulcanism or asteroid impacts, which makes the comparison hard, but climate changes that are literally an order of magnitude slower than what we’re seeing right now result in mass extinctions.

            We’ll survive, sure. The biosphere will survive, sure. But I’m not sure killing off 10% of all species on Earth is an ethical idea or a smart move.

            If CO2 were the limiting factor on plant growth, we would expect certain changes in the Mauna Loa CO2 record. For example, we might expect the pronounced and obvious seasonal swings we see in it to get larger (because there’s more plant growth in spring, and so more plants dying in winter). We would maybe expect the curve to flatten out, because a higher proportion of our emissions is getting locked up in plants.

            Neither of those things are statistically detectable in the CO2 data.

          • Nathan says:

            @ Anonymous

            Er, reducing carbon dioxide emissions is good for air quality reasons? We are nowhere remotely near the point of CO2 concentrations having a health impact.

            @ James

            We probably differ a fair amount in terms of the range of possible outcomes we’d accept as probable, but I’m at least glad we agree that the climate change = DOOM story comes with a big asterisk.

          • Anonymous says:



            I meant undesirable emissions in general. Designing filters that catch everything except CO2 sounds like it would be more costly than just having general filters.

          • James Picone says:

            @Nathan: I think he was getting at the thing where coal power plants kill a fair few people just from particulates, and phasing them out and/or sticking filters on them will reduce that.

            EDIT: And while I’m happy to agree that the media is incredibly awful at handling climate-change-related stories, and that some environmental organisations are also awful, I don’t think the push for a pricing mechanism for carbon is driven only by them – I think it genuinely comes from groups like the IPCC and actual scientists. And I don’t see so much claims of human extinction from the IPCC or scientists in general.

          • John Schilling says:


            Sounds like it, yes, but it isn’t so. First off, CO2 is a vapor, which means it isn’t captured by filters at all. Much of the “undesirable emissions in general” are particulates, which actually are filterable. Of the undesirable emissions which are not particulates, most are reactive vapors which can be made either harmless or solid by providing them with something to react to and maybe giving them a catalytic nudge. Not really a “filter”, but maybe what you meant – and most often, the thing they need to react with is ordinary oxygen, which isn’t hard to find. CO2 is effectively inert; while there are reactions it can undergo they are energetically costly and/or require fairly exotic reagents. And finally, most “undesirable emissions in general” are present at parts-per-million quantities, whereas CO2 is about 10-20% of the exhaust stream of a typical combustion engine. If you did catch all the CO2 that came out the tailpipe of your car, where would you put it? Don’t forget the energy budget for any compressors or reactors your plan requires.

            This is a pet peeve of mine, a sort of magical thinking where “pollution is bad” and so all forms of pollution are interchangeably bad. NOX is Bad Air Pollution(tm) and particulates are Bad Air Pollution(tm) and CO2 is Bad Air Pollution(tm), therefore any Good Green Anti-Pollution Efforts will reduce NOX and particulates and CO2 alike. Which, no.

          • The Dividualist proposes to help poor countries injured by AGW by sterilization. I suggest that open borders would not only help them more but help us as well.

            It’s also worth remembering that the fact that a country is poor now does not necessarily mean it will be poor in fifty years. Bangladesh has been averaging a GNP growth rate of 6%. If that rate continues for fifty years, the GNP of Bangladesh will reach almost twenty times its current level.

          • Anonymous says:

            Sounds like it, yes, but it isn’t so.

            I stand corrected.


            What do you mean by “open borders”?

          • By “open borders” I mean a return to the immigration policy that the U.S. followed for most of its history, with some exceptions for east Asians starting in the late 19th century. If you can get here and are not obviously carrying a contagious disease, you are welcome to come.

    • PGD says:

      There’s a big difference between findings of an individual study in the social sciences and an overwhelming consensus of an entire hard science discipline that has made predictions that are consistently qualitatively correct for over a decade now.

      • Anonymous says:

        overwhelming consensus

        Is not the same thing as truth.

        entire hard science discipline

        Astrology is hard science now?

        predictions that are consistently qualitatively correct for over a decade now

        This doesn’t match my perception of the matter. So far as I know, the only thing about the dominant AGW stance you cannot deny is that average temperatures are rising. The various models on how CO2 and other greenhouse gases affect global temperature are plausible, but cannot be verified as correct and/or accurate independently without access to a time machine. That AGW is a humongous, immediate threat to humanity that we need to unite right now to combat is incredible.

        • James Picone says:


          What else do you think caused the rise in average temperature? Do be sure to come up with something that has just as much experimental and theoretical backing and isn’t contradicted by, for example, measurements of total solar insolation, measurements of ocean heat content, or the second law of thermodynamics.

          • Anonymous says:

            Did I not say the mainstream theories are plausible? You seem to be reading my comment to mean that they’re not, somehow.

          • James Picone says:

            ‘plausible’ is a bit weak for ‘literally the only game in town’.

            (Also you’re still generally wrong – enhanced-greenhouse-effect warming is distinguishable from other forms of warming by, for example, stratospheric cooling, changes in the IR spectrum of Earth as seen from space, increased downwelling IR as measured by pointing a sensor at the sky, no detectable trend in the other obvious sources of warming with detectable trend in CO2, trend in atmospheric water vapour content, the regional pattern of warming (greenhouse warming warms the poles more than the tropics, warms nights more than days, and warms winters more than summers), paleoclimate data indicating that increased CO2 and warming often go together, greenhouse warming being the only viable explanation for other past climate changes (ice age -> not ice age transitions mostly), probably some other stuff I can’t think of right now.)

    • @Dividualist

      How do you predict the future without modelling?

  8. HeelBearCub says:

    It bothers me that Scott is this excercised about headlines.

    Website headlines have become (essentially) lies that get you to click on the article. A good percentage of the time the article has only a tangential relation to the headline. This is a well know phenomenon and is one reason for the term “click-bait”. On Slate I have begun to notice that the same article will be listed under three or four different headlines, one of which is of the “click to see this outrage” variety (when the article says nothing of the sort).

    This is, of course, bothersome, but it has little to do with the reporting of studies in particular.

    Given that a large number of comments on these types of posts show that, for some, the effect is to discourage healthy skepticism in favor of an almost crankish “don’t trust the man” attitude, I really don’t know that the cause of rationality is being served.

    This bothers me a fair amount. The attitude that seems to be cultivated (at least in some) is the kind that leads to not vaccinating your kids. You probably think this is a ludicrous statement. But it seems to me that what is being cultivated is a blanket distrust of institutions and scientific knowledge. Not rational parsimony, but a certain kind of paranoia.

    • Anonymous says:


      When the Pravda reported the Americans being ousted from Vietnam, the Soviet public by and large believed that the Americans actually won in Vietnam, because the Pravda never told the truth, so they took the opposite of what the Pravda said to be the truth.

      • HeelBearCub says:

        I can only guess how you mean this.

        But, if you are concluding that the modern media is a good match to Pravda, that is not correct.

        • Anonymous says:

          Any particular politicized outlet is a good match for the Pravda. They put out a generally self-consistent narrative, regardless of what the truth of the subject matter is.

          But there’s a bajillion of these outlets, each pushing their own version of reality, and the common reader, presented with such a wide array of mutually incompatible interpretations of supposedly the same facts, can reasonably assume that if he were to randomly select one of them, he would probably choose one that lies, by the sheer unlikelihood that the one he chose is actually the one telling the truth, in the mess of liars.

        • Nornagest says:

          Soviet-era Pravda was before my time; I was around to see the Berlin Wall fall, but too young to have been very aware of what was going on on the other side of it. But modern journalism has reached a point where I generally assume any article I see linked on Facebook is a lie, or at best technically true but so heavily spun that I can’t draw any useful conclusions from it. Not “black is white, up is down, freedom is slavery” levels of lying, but easily “what is this misinterpreting or leaving out that makes the lede utterly benign?”.

          The headlines are usually the most sensationalized part of the article, but my skepticism extends to body text. Especially, but not exclusively, if it’s in listicle format.

    • DensityDuck says:

      “It bothers me that Scott is this excercised about headlines.”

      He points out that the articles go on to support the thesis of the headlines, and that the neutral summary he provides shows up buried in the article with little or no emphasis. It’s not just the headlines.

      • HeelBearCub says:

        @Density Duck:

        He asks us to trust that this is so for Part I, not linking to the actual articles in question. In part II he appears to map “copy paste of a PR release” to “article written about study”. But not only that, the author of the psypost is, drumroll, Rice University. So he appears to be complaining not that PsyPost (whoever they are) got their analysis duty wrong, but that they let people do PR releases on their site.

        He also doesn’t mention this statement from the PR release “These stereotypical thoughts in turn led to negative evaluations of the employee as well as the organization and the products.” Given that Scott’s conclusion is “There was also no discrimination against the overweight on presentation evaluations in terms of overall evaluation, evaluation of employee, evaluation of product, or evaluation of company.” I would expect to him to have savaged that statement. He doesn’t even address it.

        Not only that, but the title of the PR release is “Overweight men just as likely as overweight women to face discrimination in retail settings”.

        It is not “Overweight men face the same amount of discrimination as overweight women”.

        There is a big difference between those two statements, and he appears to be conflating them.

    • Matt C says:

      What do you think healthy skepticism should look like?

      As others noted, Scott isn’t just talking about the headlines. In your post to Density Duck you pick over some details, but you don’t really challenge the basic idea that the articles are also misleading. Do you think Scott is wrong when he claims this?

      I didn’t think Scott was making a brand new case for journalistic inaccuracy in science reporting, so much as providing a clever and vivid illustration of what we have all already seen over and over and over. Haven’t you? You wanted to see the original articles that Scott was quoting, so it seems like you are the sort to sometimes look things up. Do you not find that digging into the details usually produces a materially different story than whatever the original source was claiming?

      It is not unhealthy skepticism to guess that a random article is inaccurate or wrong somehow, if most articles are in fact inaccurate or wrong somehow.


      Not at all. I think the anti vaxxers (and I know some) are definitely motivated by a distrust of what they see in the mainstream media and in how vaccines are discussed. But I think that distrust, while incorrectly applied, is honestly acquired.

      • HeelBearCub says:

        It’s the absence of context in the post that is bothering me. All articles are headlined in a manner that is designed to provoke reading. This is not unique to science articles. It has very little to do with science journalism, specifically.

        Scott has been on a mission to call out poor science. What he seems to be doing at the same time is building a case that science, academia, the government and all institutions in modern life are engaged in a huge con. I don’t think that is a good outcome. I don’t think that is the outcome he actually wants.

        • Matt C says:

          SSC readers ought to be able to integrate what Scott says with their own experiences. Scott isn’t trying to push the buttons of a crazed mob. We are fairly sensible people here.

          I’d hate to see every other post qualified with This Doesn’t Mean I Think Everything In Society Is A Big Con And You Shouldn’t Think That Either.

        • Nornagest says:

          I don’t see evidence for a huge con. I see evidence for a lot of small to medium-sized cons, many of them largely unconscious on the part of their perpetrators, but that shouldn’t be much of a surprise to anyone that’s aware of heuristics and biases research.

          • Cauê says:

            We need a catchy word for this kind of thing.

            Way too much time has been wasted recently in arguments about the non-existence of “conspiracies”, when the actual charges could be properly expressed in terms of ideological conformity, independent alignment of individual interests, and/or undirected spontaneous cooperation (not that the charges were always properly stated – having the word would help with that too). I think I’ve seen this problem in all sides of all recent online wars.

        • HeelBearCub says:

          @Matt C:

          I think, given the preceding comment, that makes yours sort of ironic.

      • I don’t think the “honestly acquired distrust” is limited to the media–it applies to the work of the researchers as well. I am reminded of what I think of as my loss of innocence, the discovery that academics I liked (and was working with in a very junior capacity—I would have been about 22 at the time) were willing to do deliberately dishonest work in order to get the desired result.

        I take the anti vaxxers and the like as in part the fault of people who try to use the claim of scientific authority to bully lower status people into doing/believing what they want them to.

      • DensityDuck says:

        The issue with antivaxers is not that they ask for evidence.

        The issue is that when presented with evidence, they no-true-scot it.

        • HeelBearCub says:

          I think the standard claim anti-vaxxers make is that the information can’t be trusted. It is a lie promulgated by big-vaxx, big-pharma, the government who have some reason to deny the truth.

          Note that this pattern of thinking is what is being encouraged here. The post highlights only failures, and sometimes oveesells them to make a point. The countervailing message showing successes in communication is missing.

          You don’t have to look far for an example of comments on this post denying overwhelming evidence.

          • DensityDuck says:

            “Note that this pattern of thinking is what is being encouraged here.”

            What’s being encouraged is to look at multiple sources and consider all of the information presented, rather than just the headline and the lead-in paragraph.

            It’s like the bit from Snow Crash about “intelligent people look at Christianity and see that 99% of it is bullshit, and they make the mistake of concluding that therefore 100% of it is bullshit.”

          • Matt C says:

            You didn’t quite put it this way, but you seem to think that anti-vaxxers are paranoid nutcases who are raving against mostly-imaginary enemies.

            The ones I’ve met are actually basically reasonable people. They are weird, and tend to some other kooky beliefs, but they’re not nutcases. They’re reasonably well informed about vaccines, recognize that vaccines are a question of costs and benefits, and have specific facts and not-looney arguments supporting their decision to not vaccinate.

            (Worth noting that some anti-vaxxers are actually semi-vaxxers, who get some vaccines but not others, or insist on a more dispersed vaccine schedule than the standard recommendations.)

          • HeelBearCub says:

            @Matt C:
            Have you actual done any reading up on the anti-vaccine movement? Or you offering very limited anecdotal data?

            And what do you mean by reasonable? Because the science is really, really, really clear. There was one guy playing a con, and he is the one that was believed, despite science bending over backwards to show how wrong he was.

          • HeelBearCub says:

            @Density Duck:
            As most people have neither the time, expertise nor the inclination to actually read the full papers, assuming they have access, what exactly does that mean?

            It would seem the implication is that one should simply ignore all reporting.

            There is a middle way. I don’t see Scott advocating it.

          • Matt C says:


            When I say these are basically reasonable people, I mean that you could have them as co-workers or neighbors and get along fine with them. You would not feel like you needed to avoid them or curtail your interactions with them.

            (Well, one is pretty pushy about getting her way, but it is garden variety pushiness.)

            I’m saying the people are reasonable, not that their anti-vax views are.

            That said, the anti vax views themselves are not insane. Wrong maybe, but not lunatic. “I just don’t think we know enough about the long term effects of immune system shock given to infants” rather than “the government has a secret plan to make an army of autistic super soldiers”.

            There is sometimes ranting about pharma companies and their profits. Not sure it’s any worse than what you’d get from any other group of Bernie Sanders supporters. Which these ladies all are.

            I haven’t read up on the anti vax movement generally. (No thank you.) Maybe our homeschooling friends aren’t representative. My working assumption, though, is that most anti-vaxxers are closer to our friends than some stupid-and-crazy caricature that people want to draw.

            I suspect this won’t help my case with you, but I’ll bring it up anyway. The ill-will toward anti-vaxxers reminds me a lot of the way people talk about creationists. I know a lot of creationists, and there, too, they are mostly reasonable people who are stubborn about holding an unpopular view, and who don’t deserve the vilification they get.

          • HeelBearCub says:

            @Matt C:
            Ah, well, see that is exactly my fear.

            Otherwise reasonable people get it in their heads that “X makes sense to me, and I can’t trust those people who I distrust, so I will trust this other source of info that matched my biases.”

            They of course don’t admit that to themselves. But this how you get otherwise reasonable people ignoring the very clear science.

            And this is the line of thinking that Scott is, unintentionally, actually encouraging.

          • “But this how you get otherwise reasonable people ignoring the very clear science.”

            A few decades back, the very clear science, as perceived by any layman paying attention to what the authorities said, recommended replacing butter with margarine because butter contained saturated fat. The margarine in question was hydrogenated vegetable oil, which consisted of transfats, which turned out to be much more dangerous than saturated fats. I have not seen any calculation of the excess mortality due to people following that advice but, given how big heart disease is, I would be surprised if it wasn’t at least in the hundreds of thousands.

            Or in other words, if you look at the meta-issue—should people trust what authoritative sources of information tell them the scientific consensus is—the anti-vaxers don’t look so bad. Our mechanisms for establishing scientific orthodoxy, combined with the mechanisms that filter information about science to the lay public, are not very reliable, so what the ordinary layman is told “all the scientists say” is quite likely to be false. In the particular case I mentioned, it turned out to be lethally false.

            Which is why I put a good deal of the blame for the anti-vaxers on people in high status positions who use exaggerated claims of scientific certainty to bully other people into doing things they think they should do.

          • Matt C says:


            In an open, pluralist society people will have a variety of beliefs and some of them will turn out to be wrong. Empirically, this seems to work out OK.

            And, as David Friedman points out, sometimes the correct beliefs of today are the wrong beliefs of tomorrow and vice versa.

  9. Jacobian says:

    The bad news is that media sources only report conclusions they agreed with ahead of time, the good news is that readers only read and remember the conclusions THEY agreed with ahead of time and for the same reasons, so the reporting probably doesn’t matter much.

  10. Anonymous says:

    Is it time to abolish the press yet?

  11. PGD says:

    If reporters are just going to google random studies and make shit up based on their abstracts (which it appears is what happened here), why don’t news sites develop the capacity to eliminate the middleman and just do some correlations themselves? You could write hundreds of articles based on unmined correlations from the NLSY, the General Social Survey, and other big longitudinal studies. And it would be no less reliable than the first study. And it would be more expensive, but you could do some kinds of randomized audit studies pretty easily using the resources of a newspaper (e.g. sending out resumes with different characteristics randomized and seeing the responses you got).

    Sure, the quality wouldn’t be that high and there would be a lot of data mining, but the same is true for most academic studies.

    • Randy M says:

      They probably don’t have anyone good enough at math for it to be credible.
      I’m surprised we haven’t see a headline of “Scientists report a break through in the search for the mysterious “P Value”, which has been implicated in significant effects in fields ranging from physics to psychology.”

  12. Deiseach says:

    What annoys me about that study, apart from anything else, is that they used actors in fat suits and not real fat people. I know the reason was probably trying to cut down variables, so if they sent Joe in twice (once as his real weight and once as overweight) then any negative results could be put down to “Aha, it was because of his perceived weight!”

    But wearing a fat suit is not the same as being fat. Imagine doing a similar study using blackface – yes, I can hear the howls of outrage now.

    I don’t know if fat men have it harder or easier or the same as fat women. I would have considered it slightly more acceptable for men to be fatter (as a fat person all my life, I’ve run into the problem of – for example – getting lab coats or wellingtons supplied by the place of work. “Oops we don’t have a woman’s coat in your size, you’ll have to wear a man’s coat”. Women are supposed to run to a limited set of sizes while men get a wider range of sizes that it’s presumed they’ll be. Re: the boots, I suppose the irony here is that I have the same shoe size as my father – so that’s on the large size for a woman but on the small size for a man, and that’s nothing to do with weight but with the length and width of my foot, the bones.) That’s possibly changing nowadays as body types for men become more stringent as they have been for women.

    Re: fat people being seen as unprofessional and sloppy and lazy, it’s a chicken-and-egg problem. If your clothes are ill-fitting, of course you’re going to look unprofessional. The problem is that (a) most clothing stores say there isn’t enough of a demand for the plus sizes so it’s not economically worth stocking them, or at least in a range of choices (you get – or used to get – a choice of two skirts, both polyester, both some variant on a shade of brown, and that’s if they even had them in your size in the first place) (b) manufacturers simply take the standard size and scale it up, and that doesn’t work. Your body shape and proportions are different when you’re fat. Some fat people have bigger arms/shoulders so they need wider/longer sleeves. Some have big busts and hips but smaller waists. Some have thick thighs. Some have apple shapes or ‘spare tyres’. It’s often a problem that you can get a blouse or shirt that is big enough in the bust but not in the waist, or more often that it gapes at the bust or neck, or that if it does close it simply looks uncomfortably tight. That does not give a professional appearance when you’re trying to put together office wear.

    That’s why I venture to say most fat people, like me, shop online or from a catalogue, not from a clothing store.

    As for hiring to work in retail – that depends on what image the store is trying to project. Hip, young, fashionable? Skinny young things in the latest gear only working the tills or asking “Can I help you?” Whereas somewhere like Wal-Mart will hire older, fatter people. Selling tools? Being overweight doesn’t matter there, it’s do you know the proper size spanner. Baked goods? That depends: a little bit of pudge is comfortable and reminiscent of granny’s lap, very overweight is off-putting because it associates the goods with “eat these and be fat like the whale behind the counter”.

    I can’t speak to the medical profession, but I wouldn’t be surprised if doctors were strongly encouraged not to be fat themselves; it’s a bit “mote and beam” to be telling people to diet and exercise when you look like you should be taking your own advice 🙂

    Re: microaggressions. So that’s what it’s called? But yes, when you’re fat, people feel quite free to walk up to you and tell you “Hey, why don’t you go on a diet?” or look into your shopping basket and comment on your purchases and all the rest of it.

    I mean, I’ve had the kids yelling at me in the street bit in my time. Eh. Goes with the territory. If you don’t develop a thick skin (pun not intended) you’re going to spend 60% of your time in public bursting into tears, at least when you’re young. As you get older, it gets a bit more acceptable to be heavy so people don’t make overt comments so much. Or you’ve trained yourself not to notice and at least keep the paranoia about “they’re all looking at me” deep down inside.

    RE: the simple answer is just lose the weight! Honey, I’ve been on diets since I was nine (first time I ever drew up a diet sheet for myself and told my mother I was going to stop eating so much). It hasn’t worked.

    • there is a large body (no pun intended) of literature that says diets don’t work. Most people will gravitate to a weight equilibrium based on biological and environmental factors.

      • onyomi says:

        Something I find strange about all the studies that say “diets don’t work”: would they say “exercise doesn’t work,” if they found that one gradually regained weight lost by exercising once one stopped exercising?

        • Murphy says:

          I’d say that it implies there’s more to it.

          I was very fit in my late teens. It didn’t feel like I was having to fight for it, the place I lived, the type of free time I had tended to push me towards a pretty good weight. I had to put physical effort in but it didn’t feel like mental effort.

          After my first job I had to move to a big city where commute times were about an extra 2 hours out of every day. The air is worse such that exercising outside of a gym with filtered air leaves me coughing like a chain smoker and feeling shit. I don’t have the time to prepare lunches in the evening and the decent food on offer for lunches is far less healthy.

          Unsurprisingly I’m about 3 to 4 stone heavier than I was before.

          Just putting lots of eternal mental and physical effort into reducing calorie intake would certainly reduce my weight but it wouldn’t do anything about all the factors which pushed my weight up so where before I got a healthy weight without constant real effort all those other things would now mean the equivalent of running constantly to stay in the same place.

          Would you consider a drug which you have to keep taking forever at ever increasing doses to get the same effect to be a cure or just a stopgap treatment?

    • Nornagest says:

      Oh, guys are presumed to come in a narrow range of shapes too. It’s just that, at least for business dress in the United States, those shapes are keg-shaped (marketed as “slim fit”) to pear-shaped.

      This isn’t true so much for casualwear; a lot of those brands run thin, probably because they’re designed for and modeled by young, bicoastal body Nazis. Most of American Apparel’s output fits me very well. But move into what’s euphemistically called “business casual”, and everyone assumes that since you’re wearing a white collar you must be an overweight schlub. I have to get all my office shirts made to measure or they look like I’m smuggling twelve yards of sailcloth around my waist, and the situation with slacks isn’t much better.

      Go to a department store in a major American city sometime and look at the back of the mannequins in the men’s department. Bet you anything the clothes on them are pinned back to pull in all that excess fabric.


    The behavioral sciences hare having a bit of a reproducibility crisis right now.

  14. onyomi says:

    Asking the general public to learn statistics and carefully read and evaluate the studies behind any news story that comes out is unreasonable (though maybe expecting them to be less credulous of stories they haven’t looked into might be more reasonable). Even expecting them to go on Reddit and hopefully have other people evaluate it is also too much to ask for the general public, though it probably works reasonably well for the nerdy crowd around here.

    What is more reasonable, I think, is to demand more rigorous reporting from reporters–after all, it is their job.

    • John Schilling says:

      Why is that last option any more reasonable than the prior ones? Reporters have no significant incentive to accede to your impotent demands, and nobody with the power to actually impose demands on reporters has any incentive to do so. How is it reasonable to demand a thing that isn’t going to happen?

      • onyomi says:

        Well, maybe if there were a higher demand among consumers for more rigorously fact-checked news, news organizations could afford to hire more people. So, like most things, it is ultimately the consumer’s fault for buying into sensationalist, canned news.

        I’m not sure which is harder/less likely: that consumers of news will become more demanding/discerning about the news they consume, or that they will simply start consuming news with more skepticism. Actually, the two might go hand in hand: the more consumers develop a habit of skepticism and fact-checking, the more valuable will seem a news source which is known to be more reliable.

        • Only if reliability is what the consumer is after. Most people read most news for entertainment.

          My guess is that you could find subsets of news, produced for people who expect to act on the information and be worse off it the information they act on is false, that did much better than the average. Anyone have examples?

          • Fifey says:

            Business journalism like The Economist, Financial Times, Bloomberg?

          • P. George Stewart says:

            “Trade rags” – when I was involved in the music industry, everyone read Music Week. Pretty boring, but it put everyone on the same page in discussions about trends, who owned what, what the major labels were up to, etc. Probably similar for many industries – catering, building, etc.?

        • onyomi says:

          I do wonder if, long-term, the habit of people posting and reading articles on Facebook, etc. might not contribute to the more discerning eye I hope to see, as nobody likes to be embarrassed by posting a study which is soon recognized to be bad or wrongly interpreted (of course, many perceive an argument against something someone posted on social media to be a personal attack and therefore rude, reducing the probability of both posting and of listening to such posts). Therefore, make sure to humiliate your friends who post poorly-researched articles on Facebook–it’s your civic duty!

          • jnicholas says:

            I think this has to some degree happened with matters of fact and the rise of google, wikipedia, and people’s constant access to them via smart phones. You can’t get away with talking shit in bars anymore, at least not about verifiable things like when a certain battle happened or who invented such and such a thing.

            We don’t yet have a similar structure for debatable beliefs, some place you can go to end the argument by pointing out a devastatingly strong, exhaustively complete account of the question you’re debating. Facebook and Reddit might be doing a little bit of that, and maybe will do a little bit more over time, but I think a better site could be made, and we would benefit a lot from having it.

    • The Reddit community is probably the best fact-checker out there. Often a commenter will find a flaw in the original article, or at least re-frame it differently than the headline , which is often misleading and sensationalist.

      • LCL says:

        That’s about the 15th time someone in these comments has advised to “go to reddit” for interpretation of social science studies (and perhaps pointers to interesting studies also).

        Assume I am convinced by this consensus and want to go to reddit. How does one “go to reddit” in the sense of using it for pointers to, and interpretation of, interesting social science studies? Preferably with other stuff filtered out as much as possible.

        Any tips are appreciated.

        • Google the title or headline of the paper and include the string

        • jnicholas says:

          I think I’m responsible for at least half of those ‘go to Reddit!’ comments, so let me clarify my position: I don’t think Reddit is necessarily great, on an absolute scale, at providing balanced assessments of studies or other claims, but I do think it is probably the most useful thing we currently have for that.

          So, my tips: like Grey Enlightenment said, if there’s a particular study you want to look at, google “ particular study”, which will return all Google’s hits for those keywords anywhere on Next, pay attention to the subreddit; the quality varies widely. AskHistorians is generally excellent, AskScience is generally pretty good. I can’t necessarily speak to many others, and it’ll obviously depend a lot on what your particular interests are – googling something like “subreddits for Topic of Interest” will help you find relevant communities.

          An important element to look at, too, is how much exposure a particular post has had: the more comments, the more likely it is that a fair number of cogent criticisms will be among them. Often the clear and reasonable criticisms will be upvoted to at or near the top; sometimes you’ll have to dig a bit.

          Not everything gets posted to reddit, or to a specific subreddit (you can post it, if it isn’t there already, too), and not everything that gets posted gets much response. But when both those conditions are met, there’s nowhere else I know of that can generate as much discussion, some of which is bound to be worthwhile.

        • John Schilling says:

          So, to test this theory I went and looked for reddit’s take on three technical papers on subjects I am professionally familiar with and which seem like they would be of interest to redditors.

          #1, the EMdrive reactionless thruster. Good news, reddit had a thread specifically dealing with the paper I was looking at, and linked to other discussions of the topic at reddit. Bad news, they missed the main point of the paper, which is that the author did a careful error analysis of his measuring equipment and found that the measured thrust is within the noise – but here are some things a better-funded lab might do to improve the measurements. The more general EMdrive discussions ignore the need for error analysis in the measurements, and completely fail to pick up on the fact that an EMdrive working as advertised violates conservation of energy and can be used to build a perpetual motion machine.

          Verdict: reddit fail, due to excessive enthusiasm for scientific novelty.

          #2, The VASIMR plasma thruster, allegedly capable of sending astronauts to Mars in 39 days. This time reddit didn’t have a link to the paper I was looking at, but expanding the search terms found them responding enthusiastically to a criticism of VASIMR by a noted Mars zealot. And while they weren’t entirely fair to Zubrin, their technical commentary was about right – VASIMR probably works, but no better than any other sort of ion or plasma drive, and the hype is based on handwaving an impossibly compact and powerful nuclear reactor into the system.

          Verdict: a solid reddit win

          #3: North Korean nuclear weapons and delivery systems, actually a series of papers one of which was written by yours truly. These have received enough press coverage that I was hoping reddit would have noticed, but as that isn’t the case I would at least have expected them to in some way address the question of North Korean nuclear weapons. Nope. A few brief threads that devolve almost immediately into the Buck Turgidson school of threat assessment and the long-debunked “fact” that North Korea can destroy Seoul with conventional artillery. Nothing that would really help a skeptical outsider assess the present or future capability of North Korea to actually conduct nuclear attacks, nothing addressing recent public developments in this area.

          Verdict: reddit fail, for complete lack of interest in the potential nuclear annihilation of Tokyo, Seoul, Honolulu, or San Diego.

          Go not to Reddit for counsel, for they will say both no and yes and then get bored and wander away.

          • Jaskologist says:

            +1 for actually applying science to the question at hand.

          • Douglas Knight says:

            Are you sure that is the “debunked” link you wanted to provide? To the extent that I can extract from it any numbers at all, it seems to say that artillery would kill 30k/hour, declining at 1% per hour, about 3 million total.

          • Protagoras says:

            @Douglas Knight, I don’t know where you get 30,000/hr; the paper says the attack would kill 30,000 quickly (when people were unprepared), and then the effect of a lot of different things (evacuations, people seeking cover, defensive efforts to deal with the attack, perhaps ammunition problems for the attackers, etc.) would mean subsequent casualties would be much lower. The only place it mentions a 1%/hr decline is where it suggests that that’s how quickly the North Korean artillery would be destroyed, but it definitely doesn’t say that’s the only, or the most important, factor that would cause the casualities to decline over time (looking at the linked study, it appears that people moving to artillery shelters, which Seoul apparently has, would be the main factor in reducing casualties drastically after the initial assault).

          • Douglas Knight says:

            If the important point is that people will enter shelters, then the answer to my question is: Mizokami was the wrong link because he does not mention shelters.

            I made up the “per hour” because Mizokami didn’t address that. I tried to indicate that by the phrase “to the extent that I can extract any numbers at all.” Would it be better to have said: this link contains no relevant numbers?

            Indeed, I thought it a very conservative choice. Looking at Cavazos, it seems that it should have been “per 10 minutes.” Applying to that Cavazos’s model of half the population in shelter at 3 hours and all at 12 hours, it still sounds like 1.5 million dead. But that number does not appear because Cavazos is simply uninterested in the question at hand.

          • John Schilling says:

            The Mizokami link is to the summary report which explicitly links the much more detailed Cavazos paper, with all the assumptions and calculations you could hope for. Mizokami, while short on numbers, correctly captures the essential point that North Korea could kill a fraction of a percent of Seoul’s population if it chose to devote its available artillery to that task. Perhaps I should have linked directly to Cavazos, sorry.

            And Cavazos, using his worst-plausible assumptions including 50% in shelters after three hours and 100% after 12 hours, explicitly puts the limit at 80,000 dead over an entire week of bombardment. You missed the part where the North Korean artillery can’t sustain the ten-minute rate of fire for twenty-four hours, and the attrition of the artillery and the dud rate and the need to keep some tubes available for military targets and all the rest. Exactly the sort of crap study-reading we’re complaining about here, cherry-picking a few factoids that caught your attention or support your contention.

            Cavazos, for his part, misses the ubiquity of shelters in Seoul. I don’t know if he ever visited the city; I was there last week. It never took me more than ten minutes to find an underground passage when I looked, and often even when I didn’t. It isn’t just subways, much of Seoul’s pedestrian walkways and shopping areas have been deliberately moved underground and linked together.

          • jnicholas says:

            Thanks for testing the idea and giving some data on the question. But I interpret your findings a little differently – perhaps because my assertion is different than the one you were responding to. I assert that Reddit is *not* great at giving complete and reliable responses to given claims, but that it is still the best available resource for laypersons with no specific knowledge in a domain to use as a potential check on claims that might be misleading.

            Your findings seem to be in line with my claim: Reddit does sometimes, but not always, give useful counterpoints that I wouldn’t have come up with on my own. If Reddit is the best we currently have, and if we can remember that Reddit’s pronouncements are not necessarily reliable or complete, but still sometimes useful, then we are better off checking Reddit’s interpretation of a claim than not checking.

            How much better off is hard to say. But your experiment might have more data to offer: what subreddits did you find your studies on, and how many comments/upvotes were there on the posts you looked at? I expect some subreddits will consistently give better output than others, and I expect that higher visibility (as measured by number of comments/upvotes) will tend to give better output than lower.

          • John Schilling says:

            If the “best available resource for laypersons” has a 50/50 track record on the question, “Is this thing being studied basically for real?”, then I submit that the best available resource for laypersons is a coin toss. As accurate as Reddit, but faster and cheaper. Also, the coin-toss always gives an answer, whereas Reddit only does so 2/3 of the time.

            If the value of Reddit is contingent on maintaining a list of reliable and unreliable subreddits, doing a quantitative comment/upvote analysis, and independently following up on counterpoints raised by redditors but a priori only 50% likely to be valid, then I’m not seeing the advantage to just reading the study. Because, yes, checking Reddit is better than not, but reading the original study is better than either and probably no more tedious than checking Reddit to your standards.

            To be useful, Reddit – or anything else aspiring to fill this role – needs to provide a five-minute answer that is significantly better than chance or than standard journalism. And five minutes may be stretching it.

          • jnicholas says:

            Coin tosses can’t give me knowledge I don’t already have, can’t explain why a given claim might not be for real, and can’t add to my future ability to discern for myself the flaws and/or strengths of similar claims. Reddit, sometimes, can. That’s useful.

            You’re right that it’s not always, or even often, easy to find or reasonably verify that information on Reddit, and that for many questions the cost/benefit evaluation will result in a “Meh, not worth it”. Like I say, I’m not arguing that Reddit is great at providing me with this information; I think we could build something that would be a massive improvement, in this respect, over Reddit and everything else that currently exists. But we haven’t built that yet, and when I strongly want to know more about what a study really means and what counterarguments to it might exist, Reddit is my best current bet.

      • Nornagest says:

        Reddit has a better hit rate than the average of the news media it’s drawing from, but that’s a really low bar to clear. Half the time the top places all go to ridiculous wank and the point that invalidates the whole thing is buried halfway down the page in a short, medium-karma comment; more than half, if the link’s about one of Reddit’s obsessions (privacy issues, science as in IFL Science, Bernie Sanders) or those of the subreddit in question (race, gender, warfare, colonialism).

        There usually is such a comment, at least, but ain’t no one got time to read halfway through a popular Reddit thread every time some idiot on Facebook links a sensationalist article.

    • Sastan says:

      The lazier and more effective method is to spread extreme cynicism about “journalists”

      One maxim of reporting is that when you are interviewing someone, you should be thinking “who is this lying liar and what is he lying about?”

      That’s how I view the news. Every single story is a very lazy, very stupid lie propagated by an infantile, egotistical political partisan. And the more you know about the subject matter, the easier it is to spot.

      Stop expecting anything approaching the truth and your consumption of “news” becomes much more enjoyable. I read it pretty much exclusively to see what the two sides are pushing as narrative.

      • LCL says:

        The big problem with the media isn’t really pushing ideological views on the news stories. That’s kind of par for the course historically and most of it is “spin” (which I think is inevitable) rather than outright lying.

        The big problem IMO is deciding what becomes a news story. Scott on that topic. Just being cynical about the news stories isn’t that big a help, because the really crucial, hard question is “what’s going unreported?”

      • Deiseach says:

        Trouble is, you can often tell when an “article” is no such thing, it’s a press release regurgitated wholescale.

        I don’t know if it’s because of time pressure and cutting jobs in newsrooms so they need filler rather than writing stories, but you do definitely see it. I suppose that’s great from the point of view of the client – that their press release and hence their “our product/study is fantastic!” is getting out there unchallenged or without investigation – but it’s not so great for the sake of journalism.

        At least you have evidence to show your clients that you’re earning the money they pay you, Mark 🙂

      • Murphy says:

        Have you ever been tempted to pepper a press release with a few zero width spaces or similar non-printing characters so that you can know when literal copy-pasting has been happening?

        Actually, testing whether zero width spaces get included in this forum:


        Edit: test successful, this forum does not filter out ZWS’s.

        Years ago in an online game I played with guilds the leadership used fingerprinted messages containing zero width spaces to root out a spy. (An enemy group liked to publicly post snippets of internal communication to sow mistrust)

        • Murphy says:

          Fair enough though it sounds odd to describe it as getting “caught”.

          The character is explicitly designed for formatting text but just isn’t normally used and hence works as a fingerprint.

          It would be like fearing for your job for using an unusual LF/CR combo in your text or other triviality. Even if someone noticed it the only way it would ever get any attention is if you took someone to court or crowed about it.

          I’d only be inclined to fingerprint text out of curiosity. Not as any kind of “sting”.

  15. Mark V Anderson says:

    I’ve always said that one should never base any conclusions on just one social science study, because of the problems inherent in these studies. As has been discussed a bit in the comments already. I think you probably need 10-20 studies that say pretty much the same thing before it should affects anyone’s judgment.

    Even for hard science I would be skeptical about the results of just one study, until its been replicated a few times (remember cold fusion?).

    • Murphy says:

      Even numbers alone don’t get you there. You need to have some idea how many studies were done but went unpublished because if 100 studies are done and only 20 of them are published then you could still have a very distorted view of it.

  16. RCF says:

    “an unblinded subjective outcome practically designed to produce placebo effects.”

    I don’t think that’s correct use of the term “placebo”. And they easily could have blinded the study; just take video from an angle at which the “fat” person can’t be seen, and show the video to people rating the responses.

    Also, “professionalism” to a large degree consists of adhering to social norms. Our society has a preference for thin people. Therefore, being fat is, in some sense, by definition “unprofessional”. On top of that, it’s harder to get professional-looking clothing in large sizes.

    • Julie K says:

      But the actor knew that he was wearing a fat suit, and that might affect his behavior.

      • RCF says:

        And fat people know they’re fat, which affects their behavior.

        • Murphy says:

          Yes but if you have a thin person in the fat suit who knows they’re playing the fat guy in an experiment may not have the same changes in behavior as a fat person who knows they’re fat and thus might prompt different responses from the clerk.

          Though your suggestion is still a lot better than the way the researchers actually did it. You could use button-cams and microphones to collect video and voice of the store worker then have that assessed in a blinded fashion. Though that might be illegal in some states.

  17. Neanderthal From Mordor says:

    The interesting part is not that newspapers are wrong about what a study reported, but what they fill the blank spaces with. Is like a rorschach test.

  18. Christopher says:

    Luckily for members of the public who want to check the results themselves, most studies of this kind are easily accessible on free public databases, right?

  19. Timothy Coish says:

    Psychometrics aside, is any of psychology actually useful? I just want to say that I’m extremely ignorant of psychology here, so that question should not be read as snark, but rather legitimate inquiry.

    From the outside in it all looks totally useless to the average person. Actually it’s worse than that, it looks like robots are trying to describe humans to me.

    • Fazathra says:

      Psychometrics aside, is any of psychology actually useful? … From the outside in it all looks totally useless to the average person. Actually it’s worse than that, it looks like robots are trying to describe humans to me.

      If you want to use academic psychology to help you predict and manipulate other humans then, yes, it is pretty useless for that. You should read Dale Carnegie or something instead. If you want to understand roughly how human cognition functions, then psychology is actually quite good.

      Psychology straddles the border between hard and soft science. Generally, the closer it gets to biology the more rigorous and productive it is. We know an impressive amount about neurons and neurotransmitters. We know quite a lot about how our perceptual systems work – especially vision. Psychophysics and perception research in general is usually pretty good. We also know a fair bit about more complex systems like memory, attention, and language at both a cognitive and a neural level.

      Psychometrics and personality research is also pretty good. IQ definitely exists. The Big Five probably exists. Disorders like ADHD, schizophrenia, and psychopathy (now ASPD) almost certainly exist. The DSM-5 has its faults, but we have come a long way since Victorian hysterias and weird Freudian complexes about your mother. Most of this research is generally descriptive rather than explanatory though. We do not have a good understanding of why or how these disorders (or normal personality variation for that matter) arises. There are a bunch of theories, usually centered around the neurotransmitter of the week, but they are all pretty controversial with heaps of evidence on both sides.

      Social psychology – which is what the fat discrimination study is – is generally pretty dodgy. This isn’t that the researchers are incompetent or malicious, just that when studying other humans it is extremely hard to control for all possible confounds and biases. Even so, a lot of terminology from less-wrong and here comes from social psychology – things like the halo effect, the fundamental attribution error (which is actually just part of a much wider and more useful attribution theory), and the outgroup homogeneity bias all come from social psychology.

      I have painted a slightly rosy picture above – in reality there is quite a bit of guff and a lot of underpowered studies which make claims way greater than anything they can support, but there is also a lot of solid science and established fact. If I were to analogize the state of psychology to any other subject, it would probably be to economics: the foundational principles are solid, but because the subject area is so vast and complex, precise prediction is impossible. Trying to use psychology to predict somebody’s behavior is a bit like trying to use economics to predict tomorrow’s stock prices.

      Source: Am grad student in computational neuroscience. Read experimental psychology in undergrad.

      • Timothy Coish says:

        First of all, thank you for the response. I appreciate someone (much) more informed than me taking the time to attempt to explain. Moreover, your post triggered some clarity in my thinking here.

        I said previously that psychology appeared to be “like robots describing humans to me”, but my real complaint with psychology is that, from my outside perspective, there seem to be no tangible skills taught to people. The inability to predict and manipulate other humans wasn’t exactly what I thought I meant by that, but it got me thinking.

        If I were having problems with a friend/girlfriend/parent/coworker and needed help I think I would turn to someone I knew who was charismatic/socially-experienced, over reading the latest psychological studies, and I don’t think that anybody who took psychology in school would necessarily have an advantage over me. On the other hand, if someone I knew was suffering from any mental health problems, I would immediately take them to a doctor of some kind.

        So I guess that I would break psychology down into three separate groups:

        1) Psychiatry
        Can be used to treat mentally unhealthy people.
        2) Psychometrics
        Can be used to hire better people? (I know the military uses a type of IQ test)
        3) Social Psychology
        Can be used to overcome biases? (being extremely charitable here)

        I just looked up the Halo Effect, as well as the Fundamental Attribution Error. I’m very tempted to simply sum up Social Psychology as simply being an entire field that exists to argue that being perfectly rational is hard and you should not assume that you are perfectly rational. Which is a good sentiment, if one I’m not sure I needed.

        The economics analogy seems slightly overly-generous, when I want to predict specific companies success or failure I’ll do the work and understand that company and the market. At the same time, being able to predict certain economic trends could very well enable someone to perform better on the market. If I want to understand someones behaviour I’ll just get to know them. Having a somewhat detailed model of human behaviour that can be summed up as “people are often irrational” seems totally useless.


          Perhaps it would be worthwhile asking someone who has relationships that go well– I don’t think this is quite the same thing as being socially experienced, though perhaps it’s what you meant.

    • Sastan says:

      Psychology is a very young science. Less than 100 years since we actually started measuring things. Most of the stuff that gets headlines is little better than philosophical shamanism. There is good work going on, but mostly, we’re so far from any useful insights it’s better to keep one’s mouth shut and figure we’re laying the groundwork for some usable science in a few hundred years.

      One way of telling what works and what doesn’t is to check and see what psychological principles are being used by advertising agencies. If they’ll spend a hundred mil on something, there’s probably an effect.

      • Timothy Coish says:

        Is 100 years really all that young? The entire semi-conductor industry is barely half that old and look at the progress made there. I think I’m being slightly snarky here, since, as you pointed out, the studies that get the attention appear to be disproportionately the most flawed. Even being charitable though, you yourself admit that we’re a ways away from any useful insights.

        If marketers use social psychology and at least believe that it’s useful then that’s a good argument. I’m unconvinced that this even happens though. Seems to me that marketers use research more along the lines of “Is Britney Spears still popular among girls aged 16-23” as opposed to anything more general. If they do use more general research, I’m not convinced that it’s anything other than to cover their asses should their marketing fail. “It’s not our fault guys, we were using SCIENCE here”.

    • Urstoff says:

      Useful never seemed like a great metric to me for any type of inquiry (although I’m a philosopher, so of course I’d believe that). Instead, if we ask whether the inquiry produces causal propositions in which we are warranted in believing, then yes, psychology does plenty of that; more than economics and sociology, less than biology and chemistry (as you would expect).

      Of course, bad paradigms stick around longer in psychology than in the physical sciences (there are still Freudians and behaviorists) and new tools get overused (see: 99.9% of fMRI studies), but again, less than in economics and sociology (not to mention literary criticism).

  20. Pseudoperson Randomian says:

    Wow this blog gets lots of comments really quickly.

    I’ll just leave this here for a giggle

    • RCF says:

      My understanding is that many people has it in their RSS feed or are otherwise notified of new top level posts.

  21. PGD says:

    That first study on mother’s time is just terrible. It’s a simple self-reported observational study. Since a lot of the headline outcome findings relate to mother-reported psychological problems, it seems entirely possible that mothers who spend more time with their kids also have the opportunity to observe more psychological/behavior problems. That alone would demolish a lot of the findings. Also, if there is any tendency for mothers to spend more time with kids who need more attention because they are more troubled, that destroys the study findings totally. (E.g. what if mothers spend more time tutoring kids who are behind in school?).

    Also it doesn’t seem to control for birth cohort, so simultaneous social trends in mothers work time/time with kids and kids’ outcomes is going to show up as a finding. (Perhaps that is why the authors finds that single parenthood is associated with higher reading scores at the 10% level — see Table 3).

    The truth is that a reporter shouldn’t be using that study as a demonstration of *anything*, let alone for making a giant, counterintuitive, complicated claim like ‘time with parents doesn’t matter’. Forget what headline they use. There is increasing evidence from large-scale, very well done studies that time away from parents in institutionalized child care at a young age leads to increased aggression and behavior problems. To do a story just relying on this lazy, crappy study alone to tackle a huge issue like the relationship between parental attention and kids well-being is clickbait-related journalistic malpractice of a high order.

    Expected more from the guy who wrote ‘beware the man of one study’. Beware the journalist of one study. And this is a particularly bad study.

    • XerxesPraelor says:

      “Expected more from the guy who wrote ‘beware the man of one study’.”

      I’m not sure how you read this: Scott definitely was both criticizing the study itself and how it was represented in the news. Your first two paragraphs are probably not at all controversial.

      • PGD says:

        Fair enough, but it seems weird to spend so much time criticizing how headlines present the story when the real problem is writing an article presenting a clear story based on this study at all.

  22. Charles says:


    But isn’t it significant that people equate overweight with unprofessionalism? If being overweight means you’re automatically considered less professional, it could indicate that you’re less likely to be hired for a presentation role even if, as the study indicates, your presentation would be just as effective as a thinner person’s. To me this seems like something worth exploring, so I don’t know why you’re so dismissive. True, you could assume that people would connect professionalism with lower weight, but aren’t studies partially done to find out which of our assumptions are correct? (Arguably even the assumption that overweight people are found less attractive is worth studying, but I will admit the anecdotal evidence is pretty overwhelming on that one.)

    • satanistgoblin says:

      I think being overweight is known to be correlated with lower intelligence, which in turn, with unprofessionalism. Probably the effect would be tiny though.

      • onyomi says:

        Nowadays it is also a class thing. In the US, at least, being skinny is a subtle marker of high class because it means you have the time, energy, and money to resist fast food, work out, etc. Of course, this is the opposite of most of history, and much of the world to this day. In the US now being tan is kind of high status (so long as it’s not in the shape of a tank top), whereas in Asia it’s still low status, as it’s associated with having to work outside, which only people who are too poor or stupid to have a knowledge-based job would do.

        • Fifey says:

          Or people with high conscientiousness eat well, exercise, and are also determined enough to get high paying jobs, thus making the association non-causal.

          • onyomi says:

            It probably works that way as well: and I think when discriminating against overweight people in hiring, most people are probably not thinking: “oh, he’s fat; he must be poor/low class.” They may well be thinking, consciously or unconsciously, “he’s fat–he probably has poor self control/will power/motivation…”

            But I think it has taken a definite subtle class marker status as well. Where I live it’s quite noticeable that the poor people are fatter than the well-off.

          • andy says:

            It might go other way round too. From personal experience, I exercise more and eat better when everything is cool in job and with family.

            When it is stressful or I have too little time, I eat worst, because I have less energy to care about things like that. I just want to stop hunger, but my mind is focused more on stress then on what exact kind of food I am about to cook. Basically, food quality is additional stress factor and get cut, because there it more important stress from job/kids/whatever.

            Exercising is in similar. It is non-essential and job, family needs etc simply take precedence. Also, things that make me relax and forget the stress for a while take precedence from the sport.

            Also, unless you are living near public park or in walkable city where you can go run for free, exercising costs money. And running is more like washing dishes – pretty much any sport that is any kind of fun/pleasant costs money and time to drive far to do it.

            Basically, if you can put much effort and time into luxuries like how you eat or exercising, then it means that other parts of your life are good. That is what it was with me anyway.

          • onyomi says:

            Oh, definitely.

            And I think this kind of virtuous/vicious cycle effect is in play in many areas of life: in academia, for example, those fortunate enough to get good job early on are then given enough free time (light teaching, sabbaticals) to do research and publish. It confirms the supposition that they were, in fact, deserving of the research-heavy job when they do, in fact, produce more research than the teachers who have to teach a lot. This tends to perpetuate a kind of class system within academia, though there are, of course, some people who are genuinely a lot more interested in teaching than in research and publishing (though I’m still they’d prefer not to be treated as second-class citizens).

            Unfortunately, our evolutionary Spidey sense doesn’t care if it’s “fair” or not: we are attracted to people on the “upswing,” as we hope they might help take us with them.

          • The Anonymouse says:


            Convicts manage to get fit between bare concrete walls. Sailors manage to get fit 1500ft beneath the Atlantic in even tighter confines. The common denominator is not access to amazing resources, but rather a desire to get fit, something surprisingly few people have.

          • Nathan says:

            Convicts also have bugger all else to do.

        • Nornagest says:

          Deep tans were much more high-status twenty years ago, before widespread awareness of skin cancer risks. Now it’s more a marker of subculture than of status.

          Nerd pallor still isn’t cool, though.

          • onyomi says:

            Yeah, there has been a notable shift away from the George Hamilton look.

          • Mary says:

            Also before it became cheap to acquire a tan in mid-winter with tanning salons.

          • Deiseach says:

            Tanning salons out, but bronzers and fake tan sprays are in: the amount of these I see on sale at the local chemist! Granted, given the usual Irish summer, your only chance at a tan is either a foreign holiday or a spray bottle 🙂

          • The new thing in status is not simply thinness, it’s being muscular and thin. A good example is the new pressure on women to have six-pack abs, which are nearly impossible to achieve without genetic inheritance unless one engages in a brutal exercise and diet regimen.

        • andy says:

          I am not from USA and stereotype about fat people here is that they are more friendly and nicer to others then thin people.

          The whole “fat are lazy and bad” thing was surprising to me when I encountered it first time.

          • onyomi says:

            Well, it’s funny, but I kind of have this stereotype myself, especially about women: I have a stereotype that fat women are more likely to have a good sense of humor. I attribute this to their lack of hotness requiring them to develop a personality. Personally, if I were hiring, I’d be less negatively impacted by someone’s overweight appearance depending on the job: if the job were fashion model or athlete, then obviously it’s a negative, but it might also be a negative for investment banker, because that has an association with needing the “eye of the tiger,” so to speak, and one assumes (erroneously in many cases, I’m sure) that fat people don’t have that.

            Conversely, being a bit overweight might confer slight advantages in certain jobs like customer service, chef, and even teacher, where appearing jovial is preferable to appearing steely-eyed (from a child’s perspective, I think slightly overweight men and women appear more matronly/avuncular).

            I wonder, then, if our culture’s recent extreme valorization of youth and sexiness isn’t partially to blame for decreased fat acceptance: being fat may make you look wise or matronly or convivial, but nobody wants that anymore: everyone wants to look young and sexy forever, and for that you want abs.

            That said, I think at some point in the past few decades, the US pushed new frontiers of fatness such that the “jolly fat man” got lost somewhere. The level of fatness which used to qualify you for “jolly fat man” status has now become very unremarkable, whereas the remarkably fat people look so obviously unhealthy that it’s hard to attribute any positive qualities to them.

          • Sastan says:

            I think there’s a “cope/checkout” binary with a lot of unattractive features, weight included.

            My pet theory is that when a person cannot compete in a dominant paradigm, they can either compensate in a related field, or reject the paradigm. So, some fat people put extra work into personality, humor, various talents, thereby offsetting their handicap somewhat.

            Others figure since they can’t compete on looks, fuck it all and everyone and get very angry and bitter. Sad when this happens.

          • DensityDuck says:

            “I wonder, then, if our culture’s recent extreme valorization of youth and sexiness ”

            There is no culture in the history of primates that has not valorized youth and sexiness (or, more properly, physical vigor and fertility) so I don’t know what you’re on about here.

          • onyomi says:


            I don’t think this is entirely true.

            Sure, youth and sexiness have always been valued to some extent and in certain realms, but so too have other qualities: wit and wisdom, for example. In premodern Chinese culture, for example, there was a much stronger emphasis on respect for elders, and even the standard of “sexy” for young men was much more about displays of learning than of nice abs. In the matriarchal societies of Yunnan to this day, the oldest female relative is basically “the decider” when it comes to major decisions.

            I’m not just talking about our valuing of youth and sexiness in sex partners; I’m talking about our valuing of youth in the board room.

            Though I should note that our society is, of course, now very much older in terms of whom is given real responsibility. In the Count of Monte Cristo, written little over 150 years ago, and set about 200 years ago, for example, the protagonist is granted the position of captain of a cargo ship (i. e. is put in charge of a major investment) at the age of nineteen. This is seen by those in the book as *a little bit* young, but hardly unheard of. Nowadays a nineteen-year old isn’t trusted to head a shift at McDonalds.

            This may seem at odds with my contention that our current culture values youth more highly, but I think the opposite is true: youth has now been extended so long that people seem to want to hold onto it indefinitely, as they associate it with fun and freedom from responsibility. In years past, youth probably mostly meant being somebody else’s apprentice/peon until you were old/experienced enough to take on “adult” responsibilities.

            Being 19 today often means the fun of adulthood (sex, alcohol, no one micromanaging your daily life) without the responsibilities, whereas being 19 200 years ago just meant being near the bottom of the adult pecking order. Hence “19ishness” now has a lot more positive associations, I think.

          • Jaskologist says:

            From Aristotle’s Rhetoric:



          • Julie K says:

            Cultural ideas about what is attractive and sexy can change. There have been times in the past when plump was considered more attractive than skinny.

          • onyomi says:


            Yes, as in the premodern China case I mentioned, the physical ideal for an elite man is pasty and a little plump, as these indicate he has the means to stay in and study, rather than working in the fields. Ideals of feminine beauty have shifted back and forth, but there were definitely periods when plump was>skinny: check out these sexy double chins!

            I would say it’s about health, but it seems not to be, especially when it comes to what’s fashionable/glamorous, rather than sexy per se. For example, while it’s easy to imagine that plump was healthier than skinny in an age of scarcity, there was a fad for a kind of tubercular beauty similar to “heroine chic” during the Ming Dynasty as well, which, while very rich historically, was still not beyond food scarcity. Healthy and sexy is good, but status seems to trump it. Or maybe the shift itself was a response to increased availability of food: as soon as poor people can afford to be fat, the rich people suddenly like being skinny.

          • John Schilling says:

            Be careful not to confuse “attractive”, “sexy”, and “fashionable”. The first two may be strongly correlated, but not necessarily the third – and it’s the third that most of the obvious cultural sources will point to.

            Pornography is perhaps a good indicator for “sexy”, if you filter out the sorts whose selling point is that you get to see (convincing stand-ins for) fashionable pop-culture figures nekkid.

          • onyomi says:

            Yeah, “sexy” doesn’t vary as much as “fashionable,” certainly, which is why I always want to say to those complaining that the standards for female thinness are unreasonable: those standards are largely about fashion and are more female-enforced than male-enforced. Most straight men I know, myself included, do not particularly desire rock-hard abs in our female sexual partners. We may be able to admire the “close-hanger” look of waiffish teenage models, but that is usually a non-sexual, aesthetic kind of admiration.

            Perhaps the problem is that for women, being “sexy” is dangerously close to being “slutty,” but being “fashionable” provides dominance over other women, which women ultimately care more about, since, after all, few women have trouble finding male sex partners if they really want them.

        • Julie K says:

          If you’re high-class, it may be that your peer group disapproves of being overweight more than the peer group a lower-class person belongs to, so you experience more peer pressure to stay thin.

    • nydwracu says:

      If being overweight means you’re automatically considered less professional, your presentation would not, all else being equal, be just as effective as a thinner person’s, because if being overweight means you’re automatically considered less professional, being overweight means you’re automatically considered less professional by the audience, not just by the hiring manager.

      • XerxesPraelor says:

        Well, if it’s not a positive error just from luck, then it does show both that fat people tend to seem less professional and that their presentations are just as effective. You can either say the data just isn’t extensive enough, or try to deal with the data. Both are valid. Personally, I think it could be that people judging the presentation don’t judge people based on whether they look professional, as sometimes super-professionalness makes you seem less trustworthy, like televangelists.

      • RCF says:

        I believe that Charles is referring to the fact that the study looked at presentation effectiveness metrics such as the audience’s views towards the product being presented, and found that based on those metrics, the presentations were equally effective.

      • andy says:

        “Being professional” does not mean “being good effective worker”. My observation is that when people want others to act more professionally, they usually mean all kind of work effectiveness unrelated/loosely related things – how nice you are, which opinions you should keep for yourself and which say out loud, how you should react to insult (usually meaning swallow it), how you should leave company etc.

        “You should maintain professional behavior” does not mean you should be effective and do your job well. It means “my civility standards and social mores are better then yours, adjust to mine”.

        The right grooming, haircut and cloth style are often in that category. Professional dress and dress your customers like is not always the same. Being fat falls into grooming category.

  23. Kyrus says:

    “and so people will come up with four diametrically opposed conclusions”

    Are they really diametrically opposed? I would say at least 1,3 and 4 could go together well.

    • Godzillarissa says:

      4 claims it is proven you spend too much time. 1 says it doesn’t matter how much time you spend.

      So 1 basically says 4 is lying, because if it was all the same (as 4 claims), you couldn’t even spend too much time.

      They still aren’t diametrically opposing conclusions, but they don’t work together that well.

      • Kyrus says:

        Well 4 might also be interpreted as “it doesn’t matter how much time you spend with your children -> any time spend is too much time spend -> moms spend too much time…”. Although I didn’t read the article. But yes, my point was more about the “diametrically”.

        • Deiseach says:

          Isn’t 4 saying “Beyond a certain amount of time spent, there are no further good or bad effects, so anything over that time is wasted”?

          And since mothers still generally are the ones spending most time with the children, that comes out as “Moms spend too much time with kids” (and perhaps vaguely accusatory tone of “don’t be a stay-at-home mom! you are not helping your kids! go out and get a real job and contribute!”)

    • 27chaos says:

      The readers’ takeaways will easily be.

    • XerxesPraelor says:

      I think the idea is that there’s one study that recommends more time with children, one that recommends less time, one that says it doesn’t matter, and one that says another aspect is what matters about it. They’re pointing in different directions.

    • RCF says:

      It does seem like if the word “diametrically” is taken literally, it’s impossible for it to take more than two arguments. It’s impossible to have four variables such that each pair has correlation -1.

      Math puzzle: is it possible to have three variables, each negatively correlated with the other two?

      • Josh says:

        Yes – the results of pairs of intransitive dice.

      • hlynkacg says:

        The first thing that springs to mind is the points of a equilateral triangle.

        Any two points form a ray/plane with the 3rd point aligned with it’s normal vector. As such moving towards any one point will increase (or at least maintain) your distance from both of the others.

        This breaks down once you have more than 3 points.

        • Aron Wall says:

          Nice example, but I think it would be cleaner if phrased in terms of “inner product” of a set of vectors all being negative, rather than thinking about distances to points.

          You can do something similar with more than 3 points if you use additional dimensions. E.g. 4 points on the vertices of a tetrahedren, 5 points on the vertices of a 5-simplex a.k.a. hypertetrahedren, and so on.

          With something like a Gaussian probability distribution, this can be used to construct examples where any number N of variables are each negatively correlated with all of the others. Although the maximum negative correlation tends to get weaker as N increases.

  24. Setsize says:

    Why, yes, most matters of social perception are very difficult to study using any meaningful amount of blinding.

    And if you want to make any useful measure of a person’s biases, you use a test stimulus that is smaller than the scale’s range. A not-overpowering stimulus. That’s kind of inherent to the nature of biases, and measurement.

    It’s interesting that you think the many different measures that were taken can themselves be ranked on a scale from “subtle” to “unsubtle” though, what experiment do you propose to back up your intuitions about that?

    Also, you’ve just rediscovered why IMRAD organization exists. “Discussion” is the code for “take a giant grain of salt right now“. Half of research psychologists I know don’t read them. Which of course goes double for press releases and news articles.

    • RCF says:

      Actually, many areas allow for blinding. I discussed below how this study could have been blinded. If the effect can’t be blinded, that suggests (doesn’t prove, but suggests) that perhaps what you’re discussing is actually incoherent.

  25. FishFinger says:

    What if there was a website that summarised academic studies and verified the trustworthiness of their reporting in the mainstream media, as a kind of Snopes?

    Is there?

  26. FacelessCraven says:


    Sounds pretty similar to “bullshedding”, eg claiming false claims are still valuable because they “shed light on this important issue”.

    • Adam Casey says:

      Oh god “awareness”. To read some of the left you’d think the only reason anything bad happens in the world is that some people don’t know about it.

      • XerxesPraelor says:

        Can we please not make fun of people for sarcastic remarks meant to satirize them?

      • RCF says:

        And given that one can steelman “I’ll pray for you” as “I’ll make a conscious effort to concentrate my awareness on your plight”, perhaps the Left shouldn’t be so dismissive of such sentiments?

        • birdboy2000 says:

          Or alternatively it’s a reason to be far more dismissive of raising awareness.

        • Anonymous says:


          I like this. I’ve been thinking for a while about parallels between Christian thought and various modern liberal projects. My favorite is that “check your privilege” is basically the same as “count your blessings”. We should make a list.

          • Deiseach says:

            Privilege = blessings! I have never thought of it like that before, but it matches so well.

            And it’s the same tone of intended as well-meaning, comes off as unctuous finger-wagging 🙂

          • Loquat says:

            For a few years now I’ve been regularly reading a SJ-oriented Christian blog, and it’s kind of astounding to me how the major criticisms the blogger makes of right-wing Christianity apply pretty much equally to much of his own commentariat without anyone seeming to notice. The main examples I can think of offhand:

            1) He claims right-wing Christianity in the US focuses too much on getting everyone to believe in Jesus, with little to no guidance given to new converts on how to be Christian besides “get more people to believe in Jesus” – a lot of internet Social Justice focuses on getting everyone to believe Privilege theory and gives little to no guidance to new converts on how to make society more just besides “get more people to believe Privilege theory”.

            2) He claims right-wing Christianity in the US has no rightward border and therefore doesn’t have a good way for anyone to openly reject extremists without leaving themselves open to charges of “squishy liberalism” – online SJ has no leftward border and doesn’t have a good way for anyone to openly reject extremists without leaving themselves open to charges of Privileged Shitlordism.

          • RCF says:

            I think that “check your privilege” means more than “count your blessings”. Perhaps I’m unaware of another use, but my understanding is that it mainly means “Be grateful for what you have” and “Don’t complain about the bad things about your life”. “Check your privilege” also has the meaning of “your perspective is invalid because you’re privileged”.

          • Murphy says:

            I’m not so sure. Privileges and blessings have different implications.

            In Christianity blessings imply that the omnipotent omniscient being has judged you deserving of being “blessed” with some advantage and you should feel grateful but shouldn’t reject your blessings because that would be like throwing a gift back in someones face.

            In SJ circles Privileges tend to imply that some horrible unfair part of an uncaring society has, without good reason, given you advantage over others and as such you should feel shame and if possible reject the most unfair of your Privileges.

          • Aron Wall says:

            Except, Christians are not usually encouraged to believe that we “deserve” to be blessed by God. We are taught that God is merciful and therefore likes to give people blessings that we didn’t merit in any way. This concept is called “grace”.

            But I agree with you that this is still very different from the way that the word “privilege” is used.

          • houseboatonstyx says:

            And it’s the same tone of intended as well-meaning, comes off as unctuous finger-wagging.”

            Naw, in the US both expressions have more meanings than those. Especially in Southern Baptist territory … where don’t think “Bless your heart!” is _always_ an insult.

          • Cord Shirt says:


            *Now* it does (descriptively speaking). It didn’t initially (and isn’t supposed to). It initially meant, “You’re assuming everyone has been given all the rights/privileges/relative advantages you have, and it’s making you look callously clueless/like Marie Antoinette.”

            That is–remember that false Marie Antoinette story? The one that was remembered because it so accurately illustrated a situation that annoys others? In the story, “Marie Antoinette” asked why the peasants were unhappy. She was told, because they don’t have bread to eat. And she replied, “Let them eat cake.”

            IOW: Because *she* had never lacked for food, it didn’t even occur to her that anyone could. So she *completely* missed the point of “they don’t have any bread,” and took it to mean, not that they had no *food*, but just that they had no *bread specifically*. So she wondered what their problem was–why couldn’t they just switch to another food? All because it never even occurred to her that it was possible to be *completely out of all food*.

            So like, the traditional response to this inferential distance was to hate, attack, and execute the privileged person–as happened to the real Marie Antoinette. When “check your privilege” was first coined, it was a much kinder improvement. It originally was what your *friends* told you. It meant, “Hey there, you’re accidentally looking like a jerk.”

            Part of the reason it turned into something else was…”Check your privilege” was supposed to be, well, a word to the wise. (As in, “A word to the wise is sufficient,” or to translate into postmodern English, “To the wise, even just one word is sufficient.”) See what I mean? –just saying “check your privilege” was supposed to be *enough* for the person to pause, think, and then *realize what they were missing*. Except…maybe that works among friends, but online? Among strangers? The inferential distance is too great. So Person A says “check your privilege” and Person B may try, but has no real chance of figuring out what it is that A thinks they’ve overlooked. Not with no more to go on than just that. The inferential distance turned out to be greater than expected.

            So what started out as meaning, “Word to the wise, ‘Marie Antoinette,'” turned into…(Marie replies, “What? It’s true! If you’re out of bread, you need to switch to cake! Jeez!” and the peasant replies,) “…what, you’re *still* that clueless? God, I didn’t think anyone *was* that clueless, and frankly, it hurts to hear it…look, just shut up, OK?” Which was then misunderstood by newer community members, and abused by bullies, as “Your privilege invalidates your opinion.”

          • Cauê says:

            @Cord Shirt

            A few people had a funny discussion here a few months ago about what the hell the phrase was supposed to mean, and what we had initially thought it meant. This makes it quite clear, and it makes sense.

            I also noticed that you used a wealth and class-based example, and I don’t think it’s a coincidence that it is more reasonable (although fictional, and implausible if taken at face value) than every single time I’ve seen this phrase “in the wild”.

    • Technically Not Anonymous says:

      That sounds a lot like a bingo card argument to me. “I don’t have to actually refute this claim because it pattern-matches to something my ingroup says is wrong!”

      • DrBeat says:

        “It doesn’t matter that the thing I said was a lie, it raised awareness of The Real Problem” can be rejected on much firmer grounds than just “pattern-matching”.

      • Julie K says:

        How do you distinguish recognizing a fallacy from matching a bingo-card argument?

        • Jeff H says:

          By whether your sympathies lie with the person doing it, mostly.

          (Perhaps people doing the bingo-card move are slightly more likely to just be looking for an excuse not to engage with your argument. But I’ve seen “that’s the so-and-so fallacy!” used that way too, albeit mostly by people who obviously have a poor understanding of fallacies.)

    • lunatic says:

      Care to elaborate?

      I had a read, and Anderson spoke a lot about values taking up the slack between what has been empirically demonstrated and what is then taken as true (what theory has been chosen).

      I get the impression that Anderson is somewhat friendly to this process, encouraging the use of explicitly feminist values to do the job. Where Scott’s examples touch on it, they mainly illustrate unsound conclusions being the result of “values” taking up slack.

      What connection did you see?

      • Adam says:

        I’m thinking mostly of the section on the relationship of the whole truth to partial truths. The headlines that Scott draws attention to here aren’t all of them (though some are) flat out untrue; the flexibility of interpretation here leaves them leeway to be misleading without necessarily lying.

        We can identify what’s misleading by bringing in more context, trying to get a more accurate sense of the whole truth. That involves not just reading the studies themselves, but bringing to bear familiarity with the standards (standards being normative) of evidence in these fields.

        In short (too late, I know) the problem is less with truth-value and more that there’s something (normatively) WRONG about presenting the truth the way they do.

  27. Jiro says:


    That one is just incorrect. There’s a difference between “proven to have no effect” and “no effect is proven”. Only the former allows them to make this argument.

    • XerxesPraelor says:

      There is some point at which “no effect is proven” becomes basically the same as “proven to have no effect. Maybe it hasn’t happened here yet because it’s only one study, but it’s still okay evidence.

      If a homeopathy study cannot prove it to have an effect, then it makes sense to not spend your money on it.

      The same applies here, and I’m worried you’re just reacting because it’s from Fox News?? (10% confidence this is true)

      • RCF says:

        If you have such extensive testing that any effect would ave shown up, then yes. But this study didn’t come close to that.

      • PGD says:

        It’s not OK evidence, it’s lousy evidence. It shouldn’t move your priors much. (see my other comment for why).

    • RCF says:

      Also, even if a particular statistic were proven to not be affected, that wouldn’t prove that there’s no effect. For instance, it might be the case that the correlation between time spent and outcomes is zero, but that doesn’t mean there’s no effect. And if there’s no effect in the range of time spent they studied, that doesn’t mean that there isn’t some minimum amount of time spent below which there’s an effect. Maybe it doesn’t matter whether you spend six hours a week or eight, but spending half an hour has an effect different from spending an hour.

    • Julie K says:

      It also depends whether Moms spend time with their kids (a) to achieve the specific objectives measured by the study, (b) to achieve some other objective or (c) just because they enjoy their company.

  28. Bruce Beegle says:

    A thin person in a fat suit is not the same as a fat person. The difference is small, but when the effects are small it may matter.

    • Decius says:

      Good point. Can we compare how fat people are treated to how fatsuited people are treated?

    • Faradn says:

      The difference may not be all that small. We get a lot of cues from faces.

      • A large part of human social intelligence evolved to detect facial mimics, so yes. Besides, all that insulation on the face reduces the visibility of facial mimics which makes the other person look dangerously, suspiciously unreadable. Fat poker face thing.

    • Tibor says:

      Could a slim person in a fat jacket conjure up the uncanny valley effect? Like “something’s not entirely right about this guy, I better watch out”.

      • onyomi says:

        Yeah, I have trouble believing that we really process people in fat suits the same way we process actual fat people, even assuming we don’t realize the deception. In particular, fat people get fat everywhere, including the face. I’ve seen pictures of fat suits where they add fat to the neck area too, but it still ends up looking a little weird–not the same as a “normal” fat person.

        Also relates to the fact that, generally, people who are fat in a “normal” way (if that makes any sense) are more attractive than people who are fat in a “weird” way: I’m sure everyone has seen people who seem to gain fat in “all the right places,” as well as people who mysteriously have, for example, a very big gut but relatively skinny thighs. The latter, I think, is perceived as being more unhealthy or somehow “off,” and hence less attractive, I think. People in fat suits are probably not just perceived as fat people, but as weird-looking fat people.

        • Tibor says:

          Or ill people. If something looks almost right but not quite, it seems like some kind of a disease to us, at least subconsciously.

          At the same time, I do not think the stereotype is all so wrong. I would be surprised if on average, really obese people were as productive and professional as others. Unless you have some kind of a medical condition, not getting fat, or at least not getting outright obese is easy. If you don’t have enough will to do that, it is not that big a stretch to assume you have less self-discipline and willpower in general. I am not saying this is always the case, but I am quite certain there is a correlation. Then, obviously, given the lack of other information about the person, people will judge a fat guy subjectively worse than a slim guy (especially, since in advertising, physical attractiveness of the promoter is important). I would not even call it a stereotype, because that word suggest that it is somehow bad and undesirable. I would call it a prior.

          • Julie K says:

            Have you seen Scott’s previous post on obesity and willpower?

          • Tibor says:


            No, thanks for pointing it out. I am not saying it is equally easy for everyone. Sure, some people do seem to stay thin even if they stuff themselves with processed food full of simple sugars.

            But the fact is that if you do eat fewer calories you will lose weight no matter what. Also, you do not really have to eat less (in fact, I don’t think it is a good idea), but do more exercise. Ideally anaerobic, aerobic exercise is very inefficient in terms of losing weight…anaerobic exercise helps you grow muscle tissue which increases your basal calorie consumption, so you might even actually have to start eating more. Plus you will look better. Also, working out does not require going to the gym (I do not like gyms much) or even having any workout equipment at home. Bodyweight exercises are as efficient as going to the gym, you can do it at home and it takes you some 2 to 3 (if you include taking a shower after the exercise) hours a week in total. Someone who is not capable of doing that simply does not have the willpower.

            I cook all my dinners. That means spending about 30 minutes twice a week cooking (I cook 4 portions at a time), a trivial amount of time that anybody has. You do not have to eat processed food or you do not have to eat just processed food (I do not really limit myself, but I usually like the stuff I cook better than the processed food from the supermarket…other than bacon which I use for cooking, sometimes I buy Weißwürste for a Bavarian breakfast, also chocolate and marzipan but that is about that :)).

            Now either someone does not want to do that and is ok with being obese, or he has a terrible willpower. Since I doubt that most obese (I am not saying overweight, but actually obese) people would not trade 3 hours a week for both looking and feeling healthy, I think it is a safe bet (in the absence of other information) that they have a low willpower.

            Umm…or maybe they just do some nonsense like a crazy diet (which nobody can hold for long and which only leads to gaining even more weight afterwards) while jogging or something like that. That would not be too surprising because the fitness industry bombards people with expensive stuff that they don’t need.

          • jaimeastorga2000 says:

            Have you seen Scott’s previous previous post on obesity and set points?

          • Tibor says:

            @jaimeastorga2000: Let me refine (correct slightly) my point:

            I am suggesting that by doing enough of the right kind of exercise (anaerobic if what you care about is losing fat because muscles require a lot of energy just to sustain themselves – therefore burn the fat even when you are not exercising, whereas jogging does not have this effect) and having a balanced diet (that mostly means enough protein so that that anaerobic exercise is good for something but also enough sugar and fat) is going to make you at least a decent looking body even if you drew a rather short straw in the genetic lottery (not if you have severe – and rare – diseases associated which cause obesity, but we are talking about the majority population, at least I am) AND that at the same time to do those things (have a decent diet and do enough exercise) is rather easy and cheap – it should not cost you more than 4 hours of your time a week (including shopping for food and cooking…unless you like elaborate recipes, which I do sometimes, but that has nothing to do with keeping healthy anymore) and cost you perhaps even less than not doing it. Doing bodyweight exercises at home only costs time and cooking raw ingredients is cheaper than eating out, even in the fast food chains, or eating processed meals (of course, it costs you more time, but not that much really, there are a lot of good 30 minute recipes and you can cook several portions at a time and put them into the fridge/freezer).

            It is because of that that I have little sympathy for obese people (except for those rare cases with serious diseases that I mentioned). At best, they are misguided and try to lose weight in a inefficient and hard way (drastically cutting calories and aerobic exercise, or maybe even no exercise). At worst, they are lazy (again, it costs you at most 4 hours a week and zero financial expense to do something about it) or do not care much about the way they look or their health. I am not saying anyone should force them to do otherwise (although I would like to see the insurance companies being allowed to change their prices based on this – I do not fancy sponsoring someone else’s lifestyle), but unless the people have some severe medical conditions or are completely confused by all the fitness ads which often prescribe something entirely wrong, I cannot see them as anything but lazy.

            I am also skeptical about flour being oh so horrible. In Europe, flour has always been a staple food. The difference is that the majority of people used to work in the field or doing some other heavy manual labor. They did not look like the Discobolus, but all that hard work means you have to grow muscles and the muscles require so many calories to sustain themselves that it gets really hard to get obese. Today the majority of people work in an office somewhere and then they maybe go jogging (which might be fun and good for the heart and so on, but not really very efficient at burning fat). Flour might be bad, processed foods might be bad, but the main difference between the past is not people overeating or people eating too little, but people not doing enough hard labour.

            European countries might have a much lower obesity than the US (or Canada or Mexico). But counts as a “small” coke in a fast food in the US is large in Europe. There also seems to be more social pressure in Europe not to be obese as opposed to being apologetic towards that, so people have a stronger motivation to do something about it. The prevalence of processed food is also lower and people cook more at home. But still, my point is not about a global scale “what does this country do wrong” (I do not contest the idea that you could probably cut down obesity in the US perhaps to the European levels if you convinced people to eat less processed food and make other changes in the composition of what they eat) but about a very local one “what do I do wrong”. One can have a healthier lifestyle even if one lives in the US. And the fact that it is easy and cheap to do so is why I don’t see a point in showing any understanding to the obese. It is not the society’s fault that you are obese, it is your own fault. If you don’t like people looking a bit down on you for that, well, do something about being obese. It is not a disease (in the vast majority of the cases).

            I have a feeling I might have repeated some points a couple of times in this post, so sorry for that. But hopefully, now it is clear what my point is.

          • caryatis says:

            @Tibor: Do you see how “I do x, y and z and I’m not obese” does not lead to the conclusion that “Obese people can just do x, y, and z and stop being fat”?

          • Tibor says:

            caryatis: Sure. That is why I am saying that most obese people can do that, not those who have same rare diseases or genetic conditions. 50 years ago, there was virtually no obesity. Now, in the US and neighbouring countries it is really high, in Europe it is lower. Unless something changed dramatically in terms of genetics in 2 generations (which is highly unlikely), all it takes is to do more hard work (or workout) and have a better diet (cook at home), in other words do what the people did 50 years ago. The point is not that it works for me, the point is that it worked for everybody not so long ago and that it is easy to reproduce.

            Also, I would not expect that much variation in something like this. Perhaps your metabolism can make things 1.5 as easy/hard, but not 10 times as easy/hard, probably not even twice as. If you have, say, 180 cm (about the average male height in central Europe), you cannot sustain yourself on 1000 calories a day and if you manage to eat 3000 a day and do no exercise, you will get obese no matter what your metabolism is.

      • Deiseach says:

        I’d agree there, if they tried to fake up a fat face on a thin-faced person, it would look weird and maybe even obviously false.

        The shop workers might have thought “This guy is plainly wearing a disguise, maybe he’s a shop lifter” and that explained the lack of interpersonal warmth 🙂

        • Cadie says:

          I wonder what time of year the study itself took place, and where. Presumably people in fat suits are not wearing shorts or T-shirts, since that would show the suit – they’re in long pants with long-sleeved shirts. In hot climates, that DOES come across as strange. If you go into a store in Dallas in July and you’re wearing full pants and layers of long-sleeved shirts, and it doesn’t appear to be some religious modesty thing, you stand out as weird and, especially in higher-crime areas, a potential shoplifter.

    • RCF says:

      It would be difficult to get exactly the same state. If it doesn’t weigh as much as the weight it’s simulating, that’s going to affect how the wearer moves, which people are going to pick up on, even if subconsciously. If it does weigh the same, then the wearer is going to be dealing with extra weight they’re not used to dealing with, and aren’t going to be moving the same as a fat person who is used to weighing that much.

    • Mary says:

      Obviously what we need is to run this test with lots of identical twins where one is fat.

      • Viliam says:

        Of course we cannot tell any of the twins whether they are the fat one or the thin one. That would ruin the experiment.

    • Lambert says:

      How dense is fat-suit material? I suspect the gait and movement of a fat person to differ significantly from that of a thin person in a fat suit. Even if the fat suit is a realistic weight, I suspect fat people would be more used to the forces and inertia acting on their body than a thin person who is suddenly made heavier.

      • Cadie says:

        Definitely. At my highest weight I had no problem going up stairs or walking round trip to the mall and back, but even though I’m in better shape now and can walk longer and up larger flights, if I wore a 65-pound suit underneath my clothes there’s no way I could walk briskly for an hour each way or look natural moving up stairs. Even with a higher body fat percentage, most mobile obese people still have more muscle than their low-20s-BMI counterparts, and the muscle is just where it needs to be to help them go about their daily lives with the extra weight.

    • Wait a minute says:

      So we don´t just discriminate against fat people, but also thin people in fat suits. Oh how much we suck in treating people equally..

    • The original Mr. X says:

      Plus, it sounds like they sent the same person in twice, so maybe the shop assistants recognised them and the coldness in their demeanour was because they were secretly thinking “Hey, isn’t this the guy who came in earlier? What’s he doing now dressed in this ridiculous fat-suit?”

  29. Fifey says:

    Studies show that about half of news stories have at least one error in them, often more. (This guy put together a table summarizing the literature.) So I interpret news stories as being “based on a true story”. There’s generally little incentive for reporters to get it right. I suspect the industry is largely a sham, and the only reason journalists haven’t exposed it for what it is is because, well…

    Your world is a lie!

    • Anon says:


      ― Michael Crichton

      • This leads to one of my approaches for evaluating sources of information. Find an overlap between what the source says and something you actually know and evaluate the quality of the source on that. Then use that evaluation to give you at least some idea of whether you should trust the source on things you cannot easily check.

        I became less interested in _Guns, Germs and Steel_ after reading a book review by the author of three books on saga period Iceland.

        • RCF says:

          And discount according to how much those areas of familiarity are shared by the general public.

          Care to give a your thoughts on “The Man Who Came Early”?

      • Nathan says:

        Some of it’s ignorance and deadlines, some of it’s deliberate misrepresentation. E.g. When I was a kid the Women’s Weekly magazine did a story on my family about these poor farming families enduring hardship and poverty in the drought. And yeah, the drought sucked. But they got us to stand in front of our (ancient, broken down and barely used) shed for a photo and claimed that it was where we lived. Because our actual house (which was nothing special) didn’t look poor enough or something. Another time a reporter quoting my mother in a story invented tears streaming down her face, for added emotional impact I guess. So yeah, innocent errors contribute a lot but there’s also a certain amount of intentional misrepresentation.

        • Viliam says:

          You should put some trigger warnings on texts like this. I was interviewed by newspapers twice.

          First case, I was at elementary school, and I won a local math olympiad. So a journalist made an interview with me. It went approximately like this:

          Journalist: “So, I heard you are good at math. You probably have a computer at home.”

          Viliam: “Nope.”

          Journalist: “But your parents are certainly going to buy you a computer now, right?”

          Viliam: “Look, lady, my father died, my mother is a teacher, there is no way we could ever afford such an expensive toy. Could we talk about something else?” (Note: the interview happened thirty years ago in Eastern Europe.)

          Journalist: “Well, if you mother would plan to buy you a computer on Christmas, you probably wouldn’t know about it yet, right? And you cannot be 100% sure this is not the case, right?”

          Viliam: “Uhm, I guess… everything is possible, hypothetically. But I really don’t think so.”

          This was the version published by the newspaper, approximately:


          Viliam: “Well, I don’t have one yet, but my mom hinted that she is going to buy me one for Christmas.”

          Second case, twenty years later, I walked near some pro-Palestinian demonstration (it was halfway between my part-time job and my university), and a journalist stopped me and asked me about my opinion on Palestinian terrorism.

          Viliam: “I don’t have enough information about the current conflict per se, but I wonder why is it okay when journalists talk about ‘Palestinians’. I mean, we don’t believe in the concept of collective guilt for other nations. But when we talk about Palestinians, we make it sound like literally all people there are participating in the terrorist activities; and if we disprove of the actions, we disprove of the whole nation.”

          Journalist: “That’s an interesting opinion, young man. I’d like to publish it. Can you tell me your name?”

          Viliam: “Uhm, just call me ‘Viliam’, but I’m not going to give you my last name unless we have a deal that I can read and confirm your article before printing.” (No, we couldn’t have such deal.)

          This was the version published by the newspaper, approximately:

          Viliam: “We shouldn’t criticize Palestinians.”

          Reading the comments below the article, I was really happy I didn’t provide my last name.

        • Jaskologist says:

          I’ve seen organizations that I was deeply involved with become the subject of articles around a dozen times. None of them were anything controversial or politicized, but the article always, always got something wrong. Often this was along the lines of getting somebody’s name wrong, but sometimes it was more substantial (the pastor of my church was transformed into the leader of the denomination for the entire state in one instance). “Spot the error” became a game for us all to play whenever a new article came out.

    • Deiseach says:

      To give journalists some credit (I’m trying to be fair here) most of them are not experts and are trying to write an article based on the assumption “Our readers are thick, how simple can we make it for them?”

      Then the sub-editor tries to pull a juicy, attention-grabbing headline out of the article (and if they don’t get one, the journalist must re-write the story until they do) because your readers, the thickos, only pay heed to juicy, attention-grabbing headlines.

      Attention-grabbing headlines = more sales = profit = the proprietor/shareholders are happy and you don’t get the sack 🙂

      It’s the same rationale that clickbait works on, and as Chesterton explains from his own days as a journalist/columnist:


      • baconbacon says:


        Attention grabbing headlines can be anything. “Study shockingly finds no discrimination against fat people” or “Liberal news outlets exaggerate discrimination to get viewers” etc etc. Heck if you are willing to put some effort into the story find some quotes from fat people complaining of discrimination, stuff them in there and turn your headline into “Fat people exaggerate discrimination for sympathy”. Click bait city.

  30. Zubon says:

    Concerning reading past the abstract: this study is the one I will mention at the drop of a hat for burying the lede. It is labeled and presented as a study of automated speed enforcement. Flip to page 16 and note that the intervention they used was putting the speed cameras in marked police vans in school zones. Research finds that people will slow down if they see marked police vans in school zones. That seems like a much more modest conclusion to try to sell than “automated speed enforcement works.”

    • Deiseach says:

      That sounds like a peach of an “We have the conclusion we want, now to construct a study around it” example.

      The dogs in the street know people slow down when they know the police van is out there. Round here there are several areas where the cops like to hide, and most people know them, and they slow down when they get to them even if they’ve been booting it along before (and will boot it along after they pass them).

      What next – a study on “When making tea, use boiling water”? 🙂

  31. Muga Sofer says:

    In fairness, when you started talking about the weight-discrimination study I immediately thought “oh, that study I heard about on Reddit where the only difference was non-blinded subjective ratings of ‘warmth’, right?”

    Clearly the solution is to rely on Reddit for everything, because God knows I’m not going to read past the abstract.

    • jnicholas says:


      I think this is the best current option, and it would go a long way toward correcting the problem that Scott is pointing out if everyone (or at least most people) knew that reading the Reddit comments was the most reliable way to get an accurate understanding of a study’s actual validity and import. Publishers, editors, and writers have incentives to mislead their readers, and they have the platform to do so largely without direct correction. Reddit provides a platform for disinterested commenters, or at least the aggregated countering biases of many biased commenters, to produce a reasonably neutral, perceptive account that is much quicker and easier to digest than reading the study yourself and relying solely on your own ability to interpret and critique it.

      Incidentally, I’m copying this comment to the subreddit since I’d like to encourage activity there and because I much prefer the commenting system there.

      • Adam Casey says:

        The best way to find out about anything is to post something wrong on reddit. Because damn you’ll find out why it wa wrong.

        • jnicholas says:

          “The best way to get a right answer on the internet is not to ask the question, but to post the wrong answer.”

          Definitely true. And Reddit isn’t explicitly harnessing this reliable fact of human nature, but it is benefiting from it anyway. I think we could build something that would deliberately make use of it, though.

          • Tibor says:

            Are you guys working for Reddit? :)) If so, you are doing hell of a job in advertising it 😀

            I think I might check out what all the fuss is about.

          • jnicholas says:

            Tibor – I don’t work for Reddit, but of course that’s what I would say if I was secretly working for them, so really all you can do is go check Reddit out and see if your experience of it matches my claims.

            You should check it out, though, seriously. Best experience is to make an account, which will be automatically subscribed to a set of default subreddits. You’ll want to unsubscribe from a lot of them, and find and subscribe to others that are in line with your interests – tailoring your feed there, just like on twitter, facebook, etc., is critical to making it a useful, informative media source for you.

            Hope you find some good stuff, and feel free to ask any questions you might have 🙂

        • John Schilling says:

          The best way to find out about anything is to post something wrong on reddit. Because damn you’ll find out why it was wrong.

          And if you post something accurate but controversial on reddit, you’ll still find out why it was wrong.

        • lzma says:

          I see a very large number of uncorrected false statements on Reddit. Probably less than half of the time I notice a false statement, someone provides a correction. There are probably many that I don’t notice.

          • Noumenon says:

            Those statements probably have just a few upvotes each. Posts in large subreddits are almost guaranteed to have the real story in the comments. It’s pretty safe to withhold your upvote until after checking the comments, because the top ones will usually tell you if it’s wrong.

          • jnicholas says:

            lzma: What Noumenon said. Sure, there are lots of false statements in lots of places on Reddit, but the likelihood of there being a correction to a false statement rises as the exposure of the false statement rises. Posts and comments that get seen by lots of readers are much more likely to have their falsehoods/weaknesses pointed out. The number of votes (both upvotes and downvotes) can almost be taken as an indicator of how reliably assessed a statement is, in the sense that the valid corrections to a much-seen statement are likely to have been already made in the responses to it. Of course, then there’s the problem of having to trawl through all the child comments in order to hear all the valid counterpoints, which are often thrown in together with a lot of other junk, and aren’t always easy to interpret themselves (although then you just apply the heuristic at the new, lower level, to however many layers is available).

      • Alexander Stanislaw says:

        Askscience is pretty good, but they don’t usually talk about recent results. However, I think that r/science is often low quality. Many of the highly upvoted comments seem to come from people who don’t know anything about the field, barely read the paper and think they know something because they’ve had a introductory statistics class (if that). At it worst, that subreddit is a very severe instance of Dunning Kruger. And because the commenters do usually know something about statistics (not much mind you), they can be convincing enough to an untrained eye.

        • jnicholas says:

          I definitely agree that the quality of commenting varies widely across subreddits. I see that as encouraging, though: it seems to imply that there are things that can be done to promote high quality comments. And I think that’s important, because we do need some forum where reliable assessments of things like scientific papers can be found, as an antidote to the problem Scott is highlighting in this post, and in general in order to assess any claims made in the public sphere. We need a structure that can reliably evaluate statements and find, to the best of human ability, the complex, nuanced truth of them.

          I’m not saying Reddit is great at that on an absolute scale, at all: I’m just saying they’re probably the best currently available version of it.

          • Comments that correct errors or debunk hype get up-votes, which though Reddit’ algorithm are quickly promoted to top where they readily visible. Given Reddit’s huge traffic, there will always be an expert or sleuth ready to pounce, even for very complicated or esoteric subjects.

    • TrivialGravitas says:

      My experience with trying to correct bullshit science reporting on Reddit is that the hivemind will upvote what it wants to believe, not good corrections. This isn’t always straightforward, sometimes it wants to believe the scientists are incompetent and/or out to get them.

      • Alexander Stanislaw says:

        Precisely, multiple sources of bias do not cancel each other out.

      • jnicholas says:

        The hivemind definitely exists in many places on Reddit, but I don’t think it’s universal, or a necessary component of any large forum. But it’s not easily avoided, either. I think avoiding it requires: 1) a mixed audience – not solely, or a large majority of, a single tribe; 2) a group that includes some people who do know what they’re talking about; 3) and a high enough average IQ + openness quotient that some sufficient number of readers/voters will acknowledge when strong points have been made against their prior beliefs. That last is probably the hardest to find or create.

        Reddit in general isn’t deliberately trying to create those conditions, but I think some subreddits are, and some of them with success. AskHistorians is probably the best example.

        I think there exists a better structure that could be made, and that we would strongly benefit from having if it could evade the hivemind and create conditions in which reliable assessments of the true value and meaning of studies and other propositions were encouraged and promoted. But that doesn’t exist yet, and I think it’s true that Reddit is currently the best place to go to find strong(ish) interpretations of public claims such as studies. It’s a long ways from great, but it’s the best we have right now.

        • Julie K says:

          > a mixed audience – not solely, or a large majority of, a single tribe

          Isn’t Reddit mainly blue-tribe?

          • it varies…I think it’s greyish-blue with grey elements

          • jnicholas says:

            Reddit as a whole is probably blue-ish, but I don’t know how strongly. But ‘Reddit’ isn’t really a community itself – its just a name for a collection of communities, and the composition of those individual communities varies widely.

            But the ‘tribes’ as I meant it don’t have to be political. I just meant that in order to get useful point-and-counterpoint, a community needs to have a representative selection of the possible points of view that are relevant to whatever it is that they’re interested in.

            And I don’t think many subreddits are deliberately trying to create that condition, but I do think that some of them are benefiting in limited ways from its presence.

          • Grey-blue, impossible to find people seriously opposing same sex marriage and so on.

          • Whatever Happened To Anonymous says:

            My impression is that Reddit started out really blue, but has been trending towards grey ever since

          • Lyn Waters says:

            @Whatever Happened To Anonymous

            I believe you’ve got that one backwards. In the early days of reddit it was quite a grey-tribe mecca. Recent years have seen to evolve into a very solidly blue-tribe site with a few hangers on from the grey days. Most of the Ron Paul fans are gone, Sanders is the Reddit candidate who makes the front page regularly.

          • anonymous says:

            If you believe their numbers, there are way too many active users for them to be mostly blue tribe. Even leaving aside the inapplicability of blue/red/gray to non-Americans there are too many Americans for them to plausibly be mostly blue tribe (particularly given the relatively narrow age/gender profile they pull from).

            That said, I’m not sure I do believe their numbers.

          • Nornagest says:

            If you believe their numbers, there are way too many active users for [Reddit] to be mostly blue tribe…

            By user count, maybe. By post count, maybe not. By perception? Definitely not. There are strong 80/20 type effects there, and that can lead to even more outsized effects on perceived culture.

            Witness how SSC got seen as a hub for The-Ideology-That-Must-Not-Be-Named for a long time while maybe a dozen users were actually Voldemortists. They just posted a lot, and it looked like they posted even more than they did to outsiders who were used to seeing much less extreme views get pearl-clutched to death at the slightest provocation.

            (That said, I’d say Reddit’s roughly split between Blue and Gray. Blue probably has the advantage in terms of numbers, but the Gray places are very Gray.)

          • anonymous says:

            Just out of curiosity I wonder roughly what you think the size is of entire US gray tribe population.

          • Nornagest says:

            You’re going to hold that up against Reddit’s 36 million user accounts, right?

            Well, maybe. Going strictly by Scott’s tongue-in-cheek intensional definition (eats paleo, drinks Soylent, rides Uber, calls things “sportsball”) I’d be surprised to see 10K out there. But if we take that as a stereotypical member of a looser social grouping — which certainly seems to be what Scott’s going for, judging by his description of the other two tribes — there are going to be a couple orders of magnitude more people that don’t come off so stereotypical. I’m a lot more Gray than I am Red or Blue, for example, and I miss every single one of Scott’s features except the one about reading lots of blogs.

            (Those 36 million active accounts probably include a lot of bots and a lot of alts, too. If I was going to try and estimate Reddit’s population, I’d start from its ~10K active subreddits, not its account total.)

          • anonymous says:

            Partly that’s what I was going for, though as you say in terms of post count or perception it is a lot fuzzier.

            But in a larger sense, I find the question of the gray tribe, it’s (non)existence and characteristics a fascinating question for reasons I can’t quite put my finger on.

            Leaving aside Scott’s very tight definition, the larger issue to me with the gray tribe was the dark matter / light matter aspects of the blue and red tribes. If I had to put my finger on the fundamental difference between a subculture and a tribe that would be it — a tribe is large enough that it can sort of conceive of itself as a whole world.

            I guess really isolated subgroups (e.g. Amish) are kind of tricky for that taxonomy, but even in the techiest part of SF can you really assume everyone is a gray? Which is one reason the offshoot of blue thing always made more sense to me.

            FWIW 1M seems like a reasonable cap to me.

          • Nornagest says:

            SF is unusual in that it’s both a strongly Gray town and a strongly Blue town; industry is overwhelmingly dominated by Grays, but government is overwhelmingly dominated by Blues, and neither can ignore the other. It’s similar to the dynamic you see in college towns in Red country, on a larger scale.

            It’s not really possible for your life to remain unaffected by people outside your tribe; but it’s possible for Grays to be your social world and Blues a sort of malign mass of pearl-clutching hippies pressing in around it. I imagine it’s similar for Blues, substituting “privileged nerds” and “intruding in on”.

    • DanielLC says:

      That’s always been my solution. Not Reddit specifically, but in my experience sometimes if you read comments you can find someone who actually read and understood the study. The news article itself isn’t worth bothering with.

    • Richard says:

      For those who are optimistic about Reddit as a good way of finding crowd-sourced truth, note that the current #2 post on r/all (in other words, the 2nd-most upvoted post on the whole website right now) is this, a post exposing Reddit’s privacy policy change as allowing them to sell your data to advertisers. It even quotes the privacy policy and highlights the text. Those who can read legalese (or just understand negation), however, will realize that the policy says the exact opposite of what the post claims it does. Reddit states that “Except as it relates to advertisers and our ad partners, we may share information with vendors, consultants, and other service providers who need access to such information to carry out work for us;” the post misunderstands the “except” and thinks this means that advertisers and ad partners are allowed to have information shared, rather than meaning that they are explicitly excluded from having information shared with them.

      And this post got tons of upvotes; #2 on r/all is pretty significant. At least the right answer was in the comments, right? Well, kinda. At the time I’m posting this, the first comment that mentions this at all is the 7th top-level comment, below 6 other threads with a bit over 300 comments total. It might move up the ranks some later, but if it does, it’ll likely be as the post is declining from its peak level of attention.

      Going against standard internet advice, I’m a huge fan of the doctrine of “always read the comments”. It can be a great way to find the best counter-argument to what the author is saying, although that really depends on the site. But reddit is not at all a magic bullet here, and sometimes makes things worse. Reddit on the whole has a substantial bias. Subreddits often have a significantly larger bias (and sometimes in a different direction from reddit overall). Even the thread title can change things significantly; the comments on an article that confirms a left-wing concern will be totally different from the ones that confirm a right-wing concern, and neither side will even seem to acknowledge the other, even in threads right next to each other on the same subreddit. It’s great that reddit provides a comment section for things like academic papers that are otherwise hosted on sites that don’t have them, and this can be really informative. But in the end it’s still an internet comment section, most of the votes are coming from people who, like you, clicked to the comments instead of reading the link, and it’ll often get you a lot more upvotes to parrot political talking points than to address uncomfortable specifics.

  32. Decius says:

    Can we fix the law? If you cite my study and lie about the results, that harms my professional standing and is slander or libel.

    Just like if I take the current headline of NYtimes (whatever it is) and cite it as “New York times celebrates terrorist attacks, denies Holocaust”, and put the truth in a couple sentences halfway through the article.

    This isn’t bias, it isn’t honest, it isn’t right, and it shouldn’t be legal.

    • Caleb says:

      The problem with this is that, as Scott pointed out, most reports of this type don’t actually contain false statements of fact. Instead, they are editorializing according to some agenda and/or highlighting the most “interesting” parts. Doing this (while dishonest) is almost certainly “protected opinion” and the courts would be reticent to apply liability even if lawmakers gave study authors standing for these claims.

      The alternative is that courts *do* allow suits against reporters to go forward based on editorialized framing. This would give study authors essentially veto power over who reports on their studies in the media, and how. I’m not sure that’s a desirable alternative.

      • Decius says:

        >“the authors also advocate organizational efforts to combat negativity against heavy customers and potential employees…”
        Unless that statement is true, it’s a lie.

        is an outright false statement not supported by the cited source or asserted anywhere else in the article.

        • Caleb says:

          Unless that statement is true, it’s a lie.

          Not necessarily. It turns on the definition of “advocate.” While advocacy could certainly include a direct statement; e.g.: ‘we encourage organizational efforts to combat negativity against heavy customers and potential employees’ that factual occurrence does not fully encompass the scope of potential meaning. On the far end of plausible meaning, one could argue that publishing the study itself amounted to “advocacy,” since it framed the issue as identifying potential weight discrimination.

          The issue that that the statement is not a precise factual statement. It alludes to facts extant, but encompasses a wide enough area of potential meaning on the factual grounds to include many potential arrangements and interpretations thereof. In other words, opinion.


          Again, not necessarily. As Scott pointed out, the study did not significance test the gender differences. He “eyeballed” the numbers and thought they looked significant towards female discrimination. In other words, he analyzed various facts (the numbers), and formulated a conclusion. The reporters can plausibly argue they did the same, only they came to a different conclusion. Said argument may not be particularly convincing, forthright, or logical. But it would be an argument, rather than a bald factual error.

          If you don’t want to approach this problem thinking like an attorney (which is commendable), think chemistry instead: Facts are the fundamental elements. They are small, concrete, indivisible, and largely uninteresting by themselves. Arguments are molecules, the mechanistic yet creative bonding of elemental facts into more complex and useful forms. Opinions are the process of chemical reaction. Opinion contain both facts and arguments, but is not simply those things in stasis.

      • RCF says:

        Publishing your opinion is protected speech. Publishing your opinion and presenting it as something someone else said is not.


        No, it gives them veto power over dishonest reporting.

        • dcardno says:

          I think it would go further than that – in practice, it would give them veto power over interpretation of their results, which is a defacto veto over reporting, just due to the expense (and nuisance factor) of defending the suit, whether justified or not.

          • Decius says:

            Shouldn’t authors have that veto? “That’s not what my study said!” is a pretty good reason to say “You shouldn’t be allowed to say that’s what my study said.”

          • Evan Þ says:

            But, Decius, what about “I don’t want people to repeat that my study said that, so I’m going to falsely claim it didn’t”? Do you trust courts to be able to tell the two apart reliably enough for journalists to be able to count on it?

          • For a real world example of the lead author of a very prominent study trying to misrepresent what it said, see:


          • James Picone says:

            @David Friedman:

            Just checked through some of your recent posts, noticed you mentioned Tol (2009) ~2 months ago. I think you might find this blog post interesting, and in particular this paper it links to.

            tl;dr: Tol (2009) is wrong, Tol (2015) is a correction to it which /significantly/ reduces the benefits. As before, only a single point has a positive outcome for warming; it’s one of Tol’s studies.

          • James:

            The corrected version of Tol’s paper can be found (for free) at:


            If you look at Figure 1, you will observe that the change due to the correction is not very large and that it is still the case that the net effect goes negative at about two degrees.

            While it is true that, on Figure 2 of the corrected version, there is only one point that is positive, it is also true that there are only two points below two degrees, one substantially positive, one barely negative. The difference between Figure 1 and Figure 2 isn’t mainly due to the correction of the original paper, which is shown on Figure 1, but the inclusion of some new estimates, mainly (if I read it correctly) one new outlier at three degrees with a negative value about five times the average of the other five estimates at that temperature.

          • RCF says:

            Or, instead of suing them, the researchers could just kill the reporter. Is it accurate to refer to this as “veto power”?

          • RCF says:



            What about ““I don’t want people to repeat that my study said that, so I’m going to falsely claim that the reporter who reported my results raped me”? Do you trust the courts to tell apart real rape accusations, and rape accusations made to punish reporters writing unfavorable stories?

            And why would someone release a study saying X, if they didn’t want people to know that they’re claiming X? That doesn’t make any sense?

          • “And why would someone release a study saying X, if they didn’t want people to know that they’re claiming X?”

            One reason is that X is a prediction. “According to our analysis, temperature over the next decade will rise by .3°C+-.1°C.” A decade passes, temperature rise by .05°C, and the authors would prefer to conceal the fact that their theory has been falsified.

            Another reason is that the authors attempted to misrepresent their results, but without actually lying. That’s the Cook et. al. 2013 case. Their actual result was that 97% of the abstracts that expressed an opinion on the cause of warming held that humans were at least part of it but only 1.6% held that humans were the principal cause.

            To conceal that fact they combined three categories and only published the summed result–97%. They then gave the combined category ambiguous labels, such as “agrees with the consensus,” that concealed how weak the agreement was.

            But they also webbed their data, possibly a requirement of the journal. So anyone who went to the trouble of looking at the data and counting how many articles were in each category could discover that the real result was not, as widely reported, “97% of climate scientists hold that humans are the cause of warming” but rather “97% of articles expressing an opinion on the cause of warming hold that humans are at least part of the cause, but only 1.6% hold that humans are the principal cause.”

            The latter summary would have served the authors’ purposes very much less well than the former. When I posted such an article on my blog and other people echoed it, the lead author responded by attacking me for an argument I never made and ignoring the argument I actually made.

            For details see:


          • Nornagest says:


            I think that should be “explicitly held that humans were the principal cause”, or you end up looking like you’ve said that 95% of abstracts expressed the negation of that statement. In reality, it’s quite likely that a belief that strong does underlie an abstract in category 2 or 3; this data’s just insufficient to expose it. 97% likely? Maybe, maybe not. Every climate scientist I’ve ever talked to has believed it, but that’s N=1 with all the caveats.

            This is, of course, unremarkable. If we applied the same methodology to measuring, say, agreement with Darwinian evolution in the evolutionary biology community, we’d get similar results: a bare handful of direct confirmations of Darwin’s claims (usually in articles inveighing against creationism), and a very large number of papers taking some of the more interesting ones as given without taking an explicit stance on the whole theory. Of course, we could say the same of string theory, too, and for all of its main rivals. The take-home here is really that counting abstracts is bad at establishing strong consensus around a theory.

            (I reckon you could keep Hanlon’s razor a little closer to hand, too.)

          • @Nornagest:

            I agree that it is likely that many more than 1.6% of the authors believed humans were the principal cause. But 1.6% was the number that the paper (by its own report) found did.

            The distinction isn’t between what the paper found people explicitly believed and what the paper found people implicitly believed but what the paper did and didn’t find.

            Take a look at the example for category 2. The statement that greenhouse gases contribute to warming doesn’t tell us that humans are the principal cause either explicitly or implicitly–or that they are not. It leaves that question open.

            So far as Hanlon’s razor is concerned, I think the evidence of deliberate dishonesty is overwhelming:

            1. They had the numbers for how many papers were in each category. For the first three:

            Level 1 = 64
            Level 2 = 922
            Level 3 = 2910

            Can you think of an honest reason to only report the sum, thus concealing how tiny Level 1 was? And to report it as “the consensus” or “humans are causing global warming,” thus leaving it unclear whether “causing” meant “a cause” or “the cause.”

            2. In a second paper, Cook claimed that 97% was the number that the paper found for humans as the main cause. That was a flat lie, and it was one facilitated by my point 1.

            3. Cook responded, online, to my criticism (reported in another blog). His response attacked me as dishonest for an argument that appeared nowhere in my post, entirely ignored the argument (Category 1 vs 1-3) that I made.

            All of these are facts you can check for yourself–the relevant material is all at or linked to my post:


            What I find shocking about the whole episode is not that one team of people produced dishonest work–that’s not unusual. It is that nobody on their side, with the exception of Tol, has been willing to call them on it, despite the fact that the dishonesty is easily provable and Cook et. al. 2013 a high profile piece that has gotten attention from everyone from Obama on down.

            And, in the lower status world of online controversy, I find it extraordinarily difficult to get anyone on the orthodox side to admit that someone on his side lied in print, even when all the relevant information is up online, webbed by the person in question or his coauthors.

            I suggest that you read my blog post and the linked material, then either:

            1. Explain why my claim is wrong

            2. Agree that John Cook lied in print about his own work

            3. Think about why you are unwilling to do either.

          • James Picone says:

            [Tol2009 discussion]
            I’d say it’s about half-and-half the correction vs. the additional points, looking at the figures.

            It’s also worth noting that honestly it’s not a very good paper – the estimates that he’s graphing are all from essentially the same model, with slight changes to parameters. Fitting a trendline and a confidence interval to them is a joke. Particularly as the models in question are known to have a bias against extreme results in either direction.

            I find it interesting that you put up figure 1, and not figure 2, which looks a lot worse for the point you were making.

            Also, notice that there’s an inflection point around 1 degree in the corrected version. Notice that we are currently at around 1 degree above preindustrial. That is, additional warming will reduce the benefits we gain from here on out, according to Tol’s model.

            I asked it last time we had this discussion, and you never answered: what result do you think a Cook13-style paper would give for the question “Does the market determine prices?” applied to economics papers?

            The big story from Cook13 is the tiny number of papers in categories 5, 6, and 7. ~3% of papers implicitly minimise human contributions to climate change (perhaps by suggesting warming occurs via magic oscillations).

            Cook et al. went to more effort than most to build a study you could check, including uploading their data, approaching scientists to self-report scores to match against the results they got looking at abstracts, and /building a website where you can rate papers yourself and compare the results you get to their results/. They don’t deserve to be smeared like this.

            People ignore Tol’s complaints because they’re poor mathematics and statistics. Free the Tol 300!

            There’s a very easy answer for why Cook answered an argument you didn’t make, and it would be obvious if you weren’t looking for reasons to doubt global warming: He’s had to deal with legions of nitwits making ill-informed and outright false arguments about his paper, and he pattern-matched your complaints to the ones he’s used to seeing. I’d be extremely surprised if you hadn’t made the same mistake before.

            The takeaway here is that David Friedman is quite happy to commit the same ‘dishonesty’ he’s accusing Cook of, by conflating the category “Does not think climate change is principally human caused” and “Does not specifically indicate climate change is principally human caused in the abstract of their paper”.

            (But for what it’s worth I agree that it would be unwise to write specific laws to allow authors of papers to sue journalists who report on them badly; allow ordinary libel law to cover the extraordinary cases where the bad reporting is actually libelous).

          • Bernd Klein says:

            @James Picone

            “They don’t deserve to be smeared like this.”

            They made false claims. Your post does not dispute this. David’s complaint was that their data did not support their claim. It is probably true that if the authors themselves were surveyed many would support the claim that humans are the principal cause of global warming (more on that later). However, there were not surveyed. The research method used was abstract reading and categorization. That data in Cook13 does not support the 97% claims about consensus. David is very careful to make no strong assertions about the actual distribution of opinions, but rather to just state what can or cannot be supported based on Cook13 data. Science is not just about truth, but about what truths can be objectively supported based on available evidence.

            Here is some more commentary on Cook along with research by qualified researchers using surveys who estimate a 78%-84% consensus.

            Abstract reading leads to low-power studies that have limited ability to detect deviations from the null hypothesis. This is because the abstract introduces a reader to ideas in a paper rather than stake out the author’s position on climate topics. Your post seems to agree with this analysis since it implies that abstract-reading would not infer the relationship between prices and markets if used on economics paper abstracts:


            People who do not understand how to estimate consensus in a population should tread carefully because some methods (like abstract-reading) tend to get terrible results while others (certain surveys) can be quite good.

            Cook13’s methods are not useful for establishing consensus estimates. So they cannot make the bold claims stated in the paper. If we let such papers slip by then we fall trap to something pointed out by José Duarte:

            Channeling von Clausewitz, for those people science is just politics by other means.

          • @Picone:

            I cited figure 1 because it was the one I noticed and it appeared to be the relevant one. My thanks for calling figure 2 to my attention.

            So far as problems with the article, I have been arguing for years that we do not and probably cannot know enough to do what it is trying to do—estimate highly uncertain effects, positive and negative, spread out over a long and uncertain future, well enough to sign their sum. That is the same argument I made in the context of population growth forty-three years ago.

            Whether Cook et. al. are being smeared depends on whether my charge is true. Which part do you dispute?

            1. Category 1 was tiny.

            2. The data were reported in a way that concealed how tiny Category 1 was, by pooling it with 2 and 3. To find out how tiny it was, a reader had to locate the webbed data and hand count the articles.

            3. The labels for the pooled data were ambiguous, left it unclear how much causation was being claimed.

            4. In the second paper, Cook (or his coauthors) claimed that 97% was the figure for humans as the main cause, which described only category 1, which was 1.6%.

            If you agree with all of the statements, you agree that the second paper was a flat lie about the first. I would think you would also agree that the first was designed to be lied about, but perhaps not.

            If you disagree, which one is not true?


            I don’t know, but it is irrelevant. As I keep pointing out, my claim is not that 97% wildly exaggerates the fraction of researchers who believe humans are the main cause of warming–I don’t know if it does or not. My claim is that Cook lied in print about his own work, and that Cook et. al. 2013 appears designed to mislead readers about its result. Whether the conclusion it claims is wildly false or only a mild exaggeration doesn’t matter.

            And my further, and more disturbing, observation is that even people like you, on the more reasonable fringe of the argument, are unwilling to admit that one of your people lied in print about his own work however clear the evidence is.

            Why do you want to change the subject to how many climate scholars believe what instead of dealing with whether someone on your side can do demonstrably dishonest work without people on his side being willing to recognize the fact? Which of my claims 1 to 4 do you dispute? If none, why do you insist on defending Cook et. al., describing my criticism as smearing them?

            So far as your defense of Cook’s response to me, you apparently do not regard his willingness, as you interpret it, to accuse me of making a dishonest argument without checking to see if I actually made that argument, or to attack my criticism without bothering to see what it was, as a reason to doubt his honesty?

          • Picone writes:


            Would you like to support that claim? Where in what I have written, here or on my blog, did I make any assertion about how many people think climate change is principally human caused?

            I quote from my original blog post:


          • Nornagest says:


            1. Explain why my claim is wrong


            3. Think about why you are unwilling to do either.

            I’ve read both — well, I’ve read your blog post, and I skimmed the linked papers. I think that passage of Cook’s second paper was wrong about his earlier work: the data he presents doesn’t establish consensus on “primary cause”, though a different analysis of it would at least gesture in that direction. But I don’t think the evidence for deliberate deception is there. People misremember what they’ve said all the time, often in ways that inflate its strength and importance.

          • @Nornagest:

            Thanks for your response.

            Cook et. al. 2013 was a major project, and far away the highest profile thing Cook had ever done. I find it difficult to believe that, writing another article within a year, he remembered his own work so badly that he confused a 1.6% result with a 97% result.

            And, on your theory, you still have to explain:

            1. Why the original paper did not report numbers for the individual categories. Combining a small number with a large number and reporting only the sum is a standard “how to lie with statistics” trick. When the author later claims in print that the large number represents what the small actually does, treating it as an innocent error stretches the bounds of my credulity.

            2. Why Cook, responding to my criticism, attacks me as dishonest for an argument I never made and entirely ignores the argument I did make.

          • Nornagest says:


            I think the mistake takes place at a higher level than that.

            I don’t get the impression, reading Cook’s original paper, that the question he’s trying to answer is any more nuanced than “do climate scientists believe in climate change” (which he probably gets asked all the time). If that’s all you’re trying to prove, then there’s no need to report anything finer-grained than he does, and the 97% figure is totally legit.

            But Cook and Bedford deals with a stronger kind of consensus, one where we care about the degree of causation. So why the mistake? I’m not a mind reader, but from here it looks most likely that Cook assumed while writing his first paper that mentions of AGW in climate-science abstracts would imply a >50% contribution in the minds of their authors. This is perfectly ordinary typical-mind stuff, and Cook would have had no reason to be careful about it — that wasn’t what he was trying to prove. Then later, when he’s presented with a challenge where that assumption actually matters, he incorrectly cites himself as proving the stronger version — which he remembers himself as showing, but wasn’t really paying attention to at the time.

            Can’t speak for his response to you. But if someone accused me of perpetrating an evil plot when I’d in fact made a dumb mistake, I’d probably be tempted to double down too.

          • James Picone says:

            Jose Duarte is just playing the choose-your-preferred-study game. Notice that he rejects Verheggen et. al. because it includes an unknown number of non-scientists, then includes a survey of the American Meterological Society, which includes an unknown number of non-researching meteorologists. Oh, and that he seems to think Cook13 is actively fraudulent, despite it being very clear that it was a survey of abstracts.

            I disagree with 4. Categories 5/6/7 were wider than you think (For example, papers that implicitly minimise anthropogenic effects by fitting cycles to the record would end up in cat 5/6/7 depending on their abstract, because they implicitly minimise anthropogenic effects. There would have been some papers by Roy Spencer there, for example), and the things you need to believe to hold “There is an anthropogenic effect on climate” and “it is not the main cause of the recent warming” are very, very strange (There are quite literally no known forcings that can explain the rise, and trying to explain it with natural variability runs into the problem that ocean heat content is /also/ increasing, quite substantially (i.e. it’s not heat going ocean->atmosphere), and the problem that large ranges of natural variation tends to imply a sensitive climate, which is inconsistent with not getting heating of the scale we’ve seen from the amount of CO2 we’ve emitted).

            It’s eminently reasonable to infer “97% of published papers agree humans are the main cause” from the results Cook got.

            The point I’m trying to make with the economics example is not that Cook’s conclusion was correct, but that his inference was correct.


            Okay, maybe you’re a superhuman paragon of virtue. But us regular mortals occasionally skim-read something somebody said, leap to a conclusion, snap off a response, and then it turns out we misinterpreted what they typed. This is particularly common in areas where you’ve seen some particular dumb argument repeated a whole bunch and are sick and tired of it, and then you skim-read some new criticism that’s superficially similar to the dumb argument.

            I think you’re far, far too willing to ascribe malice to some perfectly ordinary behaviour and a paper that you don’t like.



            Only 1.6% included explicit quantification /in their abstract/. That’s a different claim to “only 1.6% hold that humans are the principal cause”. Pedantic? Yep. Meaningless? Yep. Roughly the same scale as the complaint you have about Cook? By my lights, yes.

          • @Picone:

            Your defense of your claim, if I understand it, is that while only 1.6% of the abstracts hold that humans are the principal cause, it is possible that more than 1.6% of the articles do. What you wrote, however, and I objected to, was:


            “Does not think.” You have not yet shown that I at any point asserted that the authors of those papers did not think humans are the principal cause, which was your claim. Hence you have not yet provided any support for the claim I objected to. I am still waiting for you to either support it or retract it.

        • Caleb says:

          To be more precise, it gives them veto power over any reporting of their work which they can plausibly frame as dishonest. At least plausibly enough to avoid an anti-SLAPP motion. (Which, in most states, isn’t hard.) This is a much broader category than those reports that are actually dishonest. It might be damn near every report, depending on the creativity of your attorney.

          • RCF says:


            Do you mean, as far as filing a case, or prevailing? To prevail in a libel case, it’s not enough for the defendant’s statements to be “plausibly” dishonest.

            My understanding is that SLAPP statutes would be largely irrelevant in such cases. SLAPP statutes are primarily concerned with discovery, and what discovery would there be in such case? A plaintiff’s case would consist of two claims:

            The defendant says I said X.
            I didn’t say X.

            Both should be rather straightforward to evaluate.

          • Caleb says:

            Veto power as in being able to credibly assert a cause of action plausibly enough to make it far enough in the litigation process that defending the claim becomes economically unfeasible. I’m more concerned with strategic lawsuits and the chilling effect rather than the small handful of fully litigated and decided suits. Even successfully mounting a motion to dismiss can be an economically ruinous proposition for all but the largest players. So what you’ll get is a disparate impact; where the small and weak are forced to fold to nearly any legal claim, while the large and powerful can more realistically defendant their statements, giving them more voice.

            The relevant portion of most anti-SLAPP statues I’m considering here are the cost and fee shifting provisions. For a small time defendant, these provisions are key to their ability to mount a rigorous defense. Once a defendant of modest means faces covering their own legal fees, their incentive is to settle regardless.

            The discovery provisions aren’t irrelevant here, either. A plaintiff is allowed a certain amount of discovery, regardless (generally) of need. This can be used strategically to harass, annoy, and increase costs. Halting discovery is another small-defendant-friendly provision that significantly increases their ability to defend against frivolous claims. A plaintiff surviving an anti-SLAPP motion has a much greater bargaining position over a defendant, and thus more control over the outcome of settlement negotiations.

        • The usual way of distinguishing between your description of what someone else said and your presentation of what he said is by putting the latter in quotation marks.

          • RCF says:

            That’s a rather oblique response. Are you saying that statements not surrounded by quote marks are immune from charges of libel?

          • I wouldn’t go that far.

            But a statement about what your article said that is not in quotation marks is saying that that is how I interpret what you said. That you disagree, or that you can argue that your words don’t really mean that, doesn’t prove I’m lying.

          • Creutzer says:

            How receptive would libel courts be, thought, to pleadings of obvious and utter failure as a speaker and reader of English as well as a journalist – in journalists?

          • Murphy says:

            I think we can now reasonably dismiss David Friedman’s arguments since he has publicly admitted, in his own words that

            “I’m lying.”

            ~David Friedman

          • Cord Shirt says:



            Sadly, that is genuinely what a pair of SJWs did to me to incite others to defenestrate me.

            …I initially wrote, “…to ‘prove’ I should be defenestrated”; but you see, saying *that* leaves me open to being quoted as having said “I should be defenestrated.”


            Of course, they didn’t begin doing this until after the statement “my words were taken out of context” had become a bingo-card staple. (I didn’t even bother trying to say that. It’s become as bingo-carded, and makes you as much of a laughingstock to say, as “Some of my best friends are—“)

            And *that* happened because people *did* abuse it. People *did* say that even when their words had *not* been taken out of context.


    • And then you (the researcher) get blacklisted and your results never reported about.

  33. E. Harding says:


    -Only the most dedicated have time for that.

    And the quote should be

    “The study results decide nothing. The study reporting decides everything.” Or maybe the headlines decide everything. Those who do the study may have some impact on how the study’s reported.

    • JakeR says:

      And most of the time, only the most educated can even understand it, or read it with a critical, analytical eye.

      • So if we don’t have time to read and assess a sufficient number of studies except in a very narrow area of interest, the question becomes how do we construct/incentivise a culture/set of institutions of reliable scientific reporting?

        • Kevin C. says:

          Well, first we might want to look at the history of the problem. Has science reporting always had this problem, or did it arise at a later date (and when did science reporting emerge, anyway)? If there was a time period when science reporting “worked”, we could use it as a guide to figuring out what may need adjusting.

          Though I suspect this problem has always existed with science reporting to a mass audience, and, with the incentives for journalists, as Deiseach has outlined below, or the reader/audience incentives Onyomi has mentioned, that the problem is intractable; beliving that science has proven X if and only if science actually has proven X will always be confined to a small, high-IQ, truth-over-tribe minority, and the “masses” will always have a biased, distorted view of what is or isn’t demonstrated by science.

          • It isn’t just science. Mechanisms for filtering information work badly for the same reason in other contexts. For example:

            Time Magazine had a story that quoted my father as saying “We are all Keynesians now.” What he actually said was “In one sense, we are all Keynesians now; in another, nobody is any longer a Keynesian.” Not a small distortion, given that he was at the time the leading critic of Keynesian macro.

            A fair number of people believe, and assert, that when the stock market crashed Herbert Hoover, in true Republican fashion, cut government spending, thus helping to bring on the Great Depression. In fact, by Hoover’s final budget, he had increased federal expenditure about 50% in nominal terms, about 100% in real terms (prices having fallen), about 200% as a share of national income (real income having fallen).

            I could offer other examples of widely believed historical factoids that are not true.

          • onyomi says:

            “Not a small distortion, given that he was at the time the leading critic of Keynesian macro.”

            Wow, that is a huge distortion, and I didn’t even know about it. Even Austrians criticizing your father will sometimes cite the (incomplete) quote as a sign that he basically capitulated to the Keynesian worldview.

            Also, goes to show that the most dangerous thing is not to be misquoted, but to be pithily misquoted.

        • My sense is that a large enough audience of people are interested in truth-seeking to make a truth-orientated set of media insitutions viable, but that nobody knows a reliable mechanism by which truth-seeking can be identified on the outside. Not saying I know what the mechanism is, but it seems like it might be a solvable problem?

          • Edward Scizorhands says:

            A lot of people are interested in truth-seeking.

            But a lot of people are also interested in bending the truth-seeking to their POV, and this group will disguise themselves as the other group.

    • Faradn says:

      “-Only the most dedicated have time for that.”

      In a lot of cases, only those willing to pay their way across the wall can do that.

    • Brn says:

      A similar thing happens with many books, as Michael Kinsley noted when editor of The New Republic in the 80s. He discovered that very few people actually read the non-fiction books whose ideas they discuss, instead getting their information from reviews of the books. His advice to would-be authors was to skip writing the books and instead write the review for the book that they would right, since many more people would be exposed to their ideas that way.

      • JJR says:

        My response to that would be to point out that writing a book would cause a larger number of reviews to be written by other people. So even if most people are getting information from reviews I still expect a book to spread an idea further.

        • Vox Imperatoris says:

          Yeah, approximately four people read The Road to Serfdom, but it certainly had a significant influence.

    • Deiseach says:

      It may even be “Those who fund the study decide everything”. If I’m looking for a grant to study the effect of eating cheddar at every meal, and the International Cheddar Producers Co-Op funds me, am I going to report “Cheddar at every meal will kill you” or “In moderate proportions, cheddar is perfectly okay”?

      I don’t have to lie to tilt the result either way; eating cheese for every meal probably is bad for you, and eating a small amount once or twice a week probably is fine. But suppose now I get funded by the Vegan Movement To Free Enslaved Cows for a follow-up study on what about eating cream cheese not cheddar – which way do you think I’m going to report my results? Or rather, which way will I write up the press release?

      • This is a good and very popular approach, the problem with most discussions is that while corporate funders are seen as doing this, government, NGO or civil founders are automatically seen as neutral. I mean, people trump it around that anti-global-warming research is funded by the oil corporations. Translation: it is a lie. Very well, but why do we think the various government funders of pro-global-warming are perfectly neutral. The problem, people who propose a similar systematic bias in funding are seen as crackpots. Despite the fact it is just as clear as the opposite: the incentive of Shell is to not have fossil fuels taxed more, the incentive of government / intelligentsia / intellectuals / reporters is to have it taxed and regulated more because they are the regulators, they are the social class who does this sort of thing and I don’t understand why does this logic gets called crackpot.

        • Murphy says:

          Diffusion vs concentration of incentives?

          If you have a final stage drug trial or something similar which represents hundreds of millions worth of investment the drug company has a massive incentive to try to distort the results. It may even face an existential threat if the trial goes badly for it. They have a massive incentive to attack the single issue.

          A University trial coordinator might have an incentive to find that a particular drug does not work, they might expect the company to need to run more trials to find a new product etc but her incentives are likely diffuse. Without good reason she’s unlikely to care about this particular drug company.

          Shell and a small number of similar companies have a lot of skin in the game. If global warming is real then they face a fairly short term existential threat from regulators and governments depending on how bad the problems caused by global warming are. They have a massive incentive to attack this single issue and muddy the waters.

          Governments have a diffuse incentive. Sure they’d like to get more money by taxing the company more heavily but there’s lots of ways they can get that much more money. They could instead raise VAT a little, or income tax, or road tax or tax companies which produce plastic or Apple etc etc etc. Their options are almost unlimited. They have very little incentive to target Shell and similar companies specially.

          Reporters and intellectuals even less so, their “social class” doesn’t really gain much from going after oil companies specially if global warming isn’t real.

          There’s enough causes in the world that they could spend their energy getting power for their “social class” by attacking problems that are real. Again, they could go after dictators for murdering people, they could go after drug companies for falsifying research etc.

          They have no special incentive to screw over oil companies, there’s so many other choices.

          You’ve got an incentive there but you’re failing to account for the size of the incentive. If you see one gram being weighted against a metric ton then saying “look! look! there’s a weight on both sides so they’re the same” is going to get you odd looks.

          • Jiro says:


            An average government bureaucrat has little incentive to target Shell specifically. But not everyone is average–some of them will, by happenstance, have bigger incentives than average and some will have smaller incentives than average. The ones with the bigger incentive will target Shell.


            They don’t gain much in the sense of directly receiving funds from carbon taxes. But there are other ways to gain. You may as well say that reporters who jumped on the child ritual abuse bndwagon in the 1980’s didn’t have anything to gain.

          • Marc Whipple says:

            Governments as a whole may have diffuse interests, but it only takes one biased governmental actor to both destroy perception of government neutrality and cause massively outsized effects.

            Great example I was debating with a friend of mine recently: he bemoans the fact that Federal research money cannot be used to do research on public health and firearms. This is because at one point the director of the CDC was a major-league hoplophobe and was encouraging studies which were perceived as very biased. Congress can’t fire the director of the CDC. The only way to make him stop was to cut off funding, and the only way to make sure he didn’t game the system was to be really, really draconian about it.

            There is a new director in place and my friend has a point in that this is a big health and public policy question. (Not sure it belongs at CDC, but that’s a separate issue.) But thanks to that one guy, such projects are banned, and discussing refunding them is radioactive. Even if we did decide to trust the CDC to be objective this time, nobody’s going to take a chance.

          • Maybe the special incentive is government research funding: sky-is-potentially-falling results get more and more funding to clarify and find out. Something similar to yellow journalism – get funding by presenting results as scary therefore interesting and important.

          • Murphy says:


            Yes, the average, and there’s a lot of bureaucrats. So why isn’t there a similar conspiracy across government and research against the makers of nylon or similar?

            Reporters jump on any old shite at a seconds notice and without any need for real data. Researchers far less so.


            Unfortunately for that thesis, the government tends to err in the opposite direction, compared to the number of lives involved they tend to sink massively disproportionate amounts of money into research on rare diseases and emotive things like terrorism vs things that are likely to kill you and rarely invest notable amounts in existential threats.

            Governments actually doing something long term about climate change would be a massive departure from their normal pattern of behavior.

          • Murphy says:


            After giving it some more thought, to be credible you’d also have to point to the actual individuals and the reason they’re biased. With the CDC thing there’s a person you can clearly point to. With global warming is there some individual who’s parents were assassinated by shell oil surveyors or something? There’s a huge number of individuals from across academia and government and just saying that sure, ya individuals can be biased doesn’t cut it since they’re biased in all different directions. Shells incentives are clear but the other side lacks many people with a clear incentive to take that particular action vs thousands of equally advantageous courses they could take for themselves.

          • John Schilling says:

            The penalties for deviation from groupthink are, while not quite as severe as having one’s relatives killed by rogue Shell employees, quite substantial. Particularly for people who need to be members in good standing of a community in order to make a living and/or maintain a positive self-image. And the incentive to go along with the groupthink points very specifically in one direction.

            As David Friedman recently pointed out, the professional climatology community is lying to itself about how uniformly and absolutely it supports the strong anthropogenic global climate change hypothesis. However we got here, we are now in the realm of climate science groupthink, and that does impose strong incentives antithetical to proper scientific inquiry.

          • Jiro says:


            Because by chance, nylon doesn’t have as many bureaucrats opposed to it. It’s certainly possible that prominent political groups would have tied their politics to nylon, and that government bureaucrats would then have benefitted from pandering; it just didn’t happen to happen that way.

            Using my previous example, why was there hysteria in the 1980’s about ritual abuse, but not about the dangers of dirty telephones? No reason whatsoever, it just happened that way.

        • @Murphy

          I can’t point to a reason for bias, beyond the incentive to say what other members of the blue tribe want to hear. But I think I can offer pretty strong evidence for the existence of bias and have done so in the case of one of the economists most involved in work on the consequences of AGW:

          • Nathan says:

            For me, reading the Climategate emails was very illuminating. Not for the exposure of misconduct (though there was a little bit of that), but as an insight into the attitudes and personalities of many of the scientists at the centre of this debate.

            I was pleased to note that some demonstrated genuine scientific attitudes of honest inquiry. My estimation of the character and integrity of Hans Von Storch and Keith Briffa rose for example, despite them being on the other side of the issue to me.

            On the other hand, individuals like Michael Mann and especially Phil Jones absolutely demonstrated a strong level of bias. For example, arguing for deliberately obfuscating a result in order to avoid giving “fuel” to sceptics, or for redefining what peer reviewed literature was in order to keep a particular sceptical paper out of the IPCC report.

            It’s not obvious to me what motivates Mann, but in Jones’ case it’s clear to see the reason why he engages in this sort of behaviour is personal pride. To him, anything that casts doubt on the consensus view is attack on him and his life’s work and he takes it personally. He’s certainly not motivated by wanting higher taxes or greater government control of the economy or whatever. By his own admission, he WANTS mitigation efforts to fail, he WANTS disastrous climate change to occur, so he can be proven right.

            Not everyone is like these guys of course. But it’s clear to me that there’s enough of them that outing yourself as a sceptic in the climate science community comes with serious professional risk. After all, these are the guys peer reviewing your papers.

    • Andrew M. Farrell says:

      I think the only solution to this may be people like Scott or Aaron Carroll reading studies and then writing blog posts or filming youtube videos where they put the results in context.

      The problem is: How do we decide who is being rigorous and who is being lazy or corrupt? Dr. Oz also provides the service of putting results in context. How do we prevent corruption in scientific journalism without getting attacked by swarms of sexually-reproducing ants?

      • This is part of the more general problem of evaluating sources of information on internal evidence. It’s a critical intellectual skill, and one that is not only not taught but anti-taught in the conventional K-12 system. The pupil is provided with two sources of information, teacher and textbook, instructed, unless the teacher is unusually good, to believe both of them, tested on and rewarded for doing so.

        One way of evaluating sources of information is to look for overlaps with what you already know. When I observe that someone making an argument about AGW does not understand how the greenhouse effect works (I’m thinking of all the web pages that offer what purports to be an experiment showing that CO2 is a ghg) I know not to trust the rest of what he says.

        Another way is by how the source presents its argument. Does it appear to be offering and responding to the strongest counterarguments? Does it fail to mention facts you know of that would be evidence against the conclusion it argues for? Does it qualify its claims in the ways you would expect of someone trying to be careful not to say anything that isn’t true? Does it use emotive language or assume facts not in evidence (“scientist X must be in the pay of the Koch brothers”), to convince the reader? You can find lots of examples on both sides of climate arguments, or any other politically/ideologically loaded dispute.

        One of the things I like about the web is that, since it’s an unfiltered medium, it rapidly becomes obvious to anyone not brain dead that the fact that something appears on a web page doesn’t mean it is true, hence that you have to learn to evaluate sources on internal evidence in order to filter any useful information out of what you read. Many people are not trying–what they are looking for is confirmation of what they already believe–but for those who are it is useful education.

        • FullMeta_Rationalist says:

          Some months ago, I read an article about Lockheed Martin’s plan to invent a commercial fusion reactor. The article I was reading sounded pretty convincing. I couldn’t find anything immediately wrong with it. But in the back of my mind, a little voice said “We’ve said ‘commercial fusion is just around the corner’ for half a century now. What makes this time different?”

          Sure enough, the reddit comment section debunked the article. According to the reddit’s professional nuclear engineers, the article failed to mention the biggest obstacle to fusion: finding a containment material strong enough to withstand the neutrinos without crumbling. I think the figure given was “we need a material 5x stronger than concrete” (or something like that). Experiences like this remind me that articles can sound really convincing to those outside a field, and still be misleading.

          Supposedly in the old days, education was structured around the trivium and quadrivium. The trivium comprised grammar (jargon), dialectic (logic/critical thinking), and rhetoric (design/application). It was a pyramid, where each layer depended on those below it. Becoming proficient in a subject meant proficiency in all three areas. Grammar and rhetoric seem unique to each subject. Dialectic includes a model, but also involves logic which can be abstracted across many subjects. Thus, the modern obsession with “critical thinking skills”.

          The issue with the Lockheed Martin article was that I had no model of how fusion works beyond “atoms smash each other”. I don’t even have the vocab to discuss it at a high level. So despite my “critical thinking skills” and “ability to sniff out contradictions”, I was unable to pinpoint why the article was unrealistically optimistic about fusion.

          This is a known failure mode among nerds. “I’m so logical! Logic will solve everything!”. And then they won’t have a clue about the subject matter, but think they will because “Logic (TM)”.

          To be able to grapple with a scientific study, the masses will need to have at least an engineer’s baseline idea of how the subject matter works. “Baseline” in the sense of an “Think fast: what obstacle does this Lockheed article conveniently omit?” One simply has to invest some knowledge, or otherwise they won’t be able to grapple.

          The reason people don’t already know how fusion works is because there’s a cost to knowledge acquisition. And the ROI is poor, because most people will never need to a working knowledge of commercial fusion reactors. So I think our options are A) make the knowledge more accessible or at least B) defeat Dunning Kruger by turning Unknown Unknown’s into Known Unknowns (“I don’t know anything about fusion, so I should remain skeptical”).

          • jnicholas says:

            I think options A and B could be complementary. If we can make knowledge as accessible as we can figure out how to make it – i.e., we break it down into the smallest coherent chunks, organize it in the clearest logical structure, and explain it with the most intuitive analogies and examples – and put it all in a Wikipedia-like central and well-known source, then everyone who wants to have an informed opinion on a question will know where to go. And if we find on going there that we can’t understand the explanation, whether we want to acknowledge it or not it should make us reluctant to discuss the subject as if we knew all about it.

          • FullMeta_Rationalist says:

            I am pessimistic about option A for several reasons.

            First, the reddit comments gave me the impression that material embrittlement wasn’t just a problem, but THE problem. E.g. one engineer said “nuclear fusion really boils down to a material engineering problem”. Before then, my understanding of fusion suggested that the primary problem was simply making the reactor more efficient. Because the issue in pop science is always phrased as “the input energy is still greater than the output energy”.

            I just went over to Wikipedia’s Fusion Power page. It mentions embrittlement. But it doesn’t put it on a pedestal. Instead, it’s tucked away into a corner. This makes me question whether wikipedia alone will give me the same experience/intuition as a professional nuclear engineer. Personally, I wouldn’t expect it to. But how many redditors have you seen fall victim to “armchair expert from wikipedia syndrome”?

            Don’t get me wrong, I’m glad to live in an age when knowledge is just a wikipedia link away. I’m not totally defeatist towards option A. But I doubt it’s totally soluble within a wikipedia framework. Also, sometimes I look up mathy subjects. I never get very far because the hyperlinks just point towards more inscrutable math pages and I have no idea where to start. Same thing always happens with mining wikipedia on the Middle East, though I’ve made limited headway recently. (On this note, can anyone explain to me what a lie algebra is? this has stumped me for ages.)

            Second, I doubt the masses even WANT to understand fusion. No matter how efficient we make rockets, traversing the distance between Earth and the Moon requires a minimum amount of energy. And no matter how many coursera’s the web offers, traversing the inferrential distance between what we know and the novel subject matter requires a minimum amount of energy. Some people simply do not want to pay this cost.

            I mean really, the media outlets are just giving people what they want to hear. That’s what profit incentivizes the media to converge upon. That’s what they really want. As a consumer, why pay for something like “nuclear engineering 101” when you just want to confirm your biases? Personally, I spend hours on Adventures In Wikipedia (TM). I also know that I am not like the average Joe (in this regard).

            Third, the education system gives us the potential to have learned advanced subject matters already; so there must be a darn good reason we don’t. I had a reputation in high school for “acing without trying”. This was particularly strong in science class. Part of the reason was because I already knew most of the material. E.g. in the 3rd(?) grade, I received for Christmas(?) a science encyclopedia. I devoured it. By the time I got to 6th(?) grade science class, I was like “of course water boils at 100 centigrade, doesn’t everybody know that?” But apparently, this was not obvious to my classmates. This continued all through high school.

            One of Scott’s posts recalls how he used to think school was dumb because “it would be so much simpler if they taught it X way”. But then he did a teaching gig, and realized that the traditional methods he hated in school were actually more effective on the majority of his students. This mirrors my own experience in that I would bug teachers about “why can’t you just say it X way?” But after reading Scott’s post (among other bloggers’ posts), I am lead to believe that not all my peers would have willingly read the encyclopedia in 3rd grade if given the chance (I couldn’t wrap my head around this for the longest time; typical mind fallacy I guess).

          • science says:

            It’s high energy neutrons they are worried about the materials holding up to, not neutrinos (which rarely interact with matter and so will mostly end up out in space).

            And I think your reddit engineer rather oversold the importance of the issue. It’s a problem, sure, but it isn’t THE problem. We don’t know how to get a high enough energy gain factor (Q) yet. And even if we did the current approach we are taking may mean that a reactor large enough to produce significant net power will be so large as to be inherently uneconomical.

            Those problems are more fundamental than the materials science advances needed to make the plants last for decades that it would need to pay off.

          • Who wouldn't want to be Anonymous says:

            This is pretty off topic, by my internal model of fusion is that the problem is largely one of [efficient] confinement. You have to smoosh the atoms together really, really hard. Using lasers to do it works… at a stunning net negative in energy.

            The other option being magnetic confinement which might be able to produce a net positive energy, but we can’t get it to work. The Tokamak style reactors we’ve been building for the last fifty years have a fundamental problem where something-something-wave function collapse something-something-blow a hole in the side of the reactor. This doesn’t sound very practical. Reactor designs that don’t have this problem are theoretically possible, but have crazy complex designs. We’re just now getting the computing power necessary to make theoretical designs that are only slightly less prone to blowing holes in themselves. Going from “slightly less likely” to “not at all” is still a huge leap in computation power… and says nothing about our ability to actually build those designs on spec.

            I actually wasn’t aware of the neutron induced enbrittlement, which I gather is more of an issue of long term maintenance costs than building a functional net positive reactor in the first place?

            Anyway, if anybody actually knows that sort of thing, is that a reasonable model to have of the situation?

          • jnicholas says:

            FullMeta_Rationalist: You make several good points about what’s imperfect about existing media sites as a means of gaining and verifying the reliability of information.

            First, that they are often incomplete – the reddit thread you read led you to believe that material embrittlement was the major obstacle to fusion, but that turned out not to be the case. The thread had viable information that was new and useful to you, but it didn’t contain *all* the information relevant to the difficulties of fusion tech, and there was no way to know *how much* of the total relevant information that is known to humans-in-the-aggregate was represented there. You could probably have assumed that your information was incomplete, but could not have known how complete a picture you possessed, or how much confidence you could assign to the bits you had.

            Second, you note that Wikipedia has no coherent, orderly meta-structure for the information it contains. If you start reading a math article, then follow links to other articles that seem to promise underlying explanations, there is no direction to your wanderings, no reliable order via which you can start out at simpler concepts and slowly build up from them to whatever heights you want to reach. If there exists some inferential chain of steps from a concept you understand to the one you want to understand, Wikipedia cannot help you efficiently trace it, except by accident. If you wander long enough, you’ll pass through all the points on the chain sooner or later – probably later, and you’ll never be sure you’ve heard everything that every other wanderer in the space thought was important.

            So I didn’t mean that Wikipedia’s format is adequate to the problem of making our collected knowledge maximally accessible. It isn’t, and Reddit isn’t either. We need something new, and it needs to permit and encourage both completeness – i.e., collecting and making permanently available all relevant human knowledge in a given domain – and logical order, such that users can begin reading at a concept they understand, and follow a reliable, efficient path from where they are to where they want to be.

            If you’re following the SSC subreddit, I’m trying to put together a compelling presentation of how we might build such a site, and I’m hoping to post it there soon looking for discussion and criticism.

          • science says:

            @Who wouldn’t

            Not terribly far off, but wave collapse has nothing to do with it. The (fiendishly difficult) modeling problem is macro scale plasma fluid dynamics in the presence of strong, dynamic EM fields.

          • Who wouldn't want to be Anonymous says:


            Thanks. Though, to be clear the wave collapse bit was supposed to be a humorous stand-in for “probably complex reasons I don’t understand.” I’m also pretty sure whatever the problem is doesn’t usually cause the reactor to literally blow up, either. Otherwise, it would be really, really hard to get the grant money to build them (and graduate students brave enough to run them).

          • FullMeta_Rationalist says:


            I’m not the person to ask about neutron embrittlement, etc. I’m just going off what I’ve read on reddit(?). But for the record, “neutrinos” was a typo. It was probably a rogue spell-check. I may not be a physicist, but I’m familiar enough with physics to understand that neutrinos are intangible for all practical purposes.



            What you describe sounds like what Metacademy is doing. I haven’t really looked at it myself. But in case you weren’t aware, you should probably check it out. If not to contribute, then to examine as a case study.

            The idea is to organize links into trees called “roadmaps”. It’s expected to allow for easier autodidacting. This contrasts with wikipedia’s cyclic graph. Last I checked, they were still in the infant phase with tons of blank pages.

            I don’t really follow reddit a lot these days. So I’m not familiar with your post. I queried the subreddit, but didn’t find anything relevent. Give me a link, and I’ll give it a look though.

          • jnicholas says:

            FM_R – I didn’t know about Metacademy, and that is something like one of the points I was making, and I’m glad to see that they exist. What they’re trying to do doesn’t solve the problem that I’m interested in, though.

            And I’m sorry if I was unclear: I haven’t posted my idea to the subreddit yet. I’m still trying to assemble a coherent and reasonably complete explanation of it, but I’m hoping to post it in a few days.

            Thanks for the discussion.

        • Julie K says:


          Good point. It was not until some point after I graduated college that I finally grasped the point of footnotes and bibliographies. Obviously, I had written plenty of reports in high school and college, but footnotes were just things that you included because the teacher demanded it. I never quite understood that there could be disagreement over the sort of facts taught in the classroom, and that references were there to provide evidence to back up your claims.

        • Deiseach says:

          I think the only solution is wide reading and an interest in history, no matter how amateur.

          You get the usual “lists of kings and battles” approach in school, but even there once you get into senior classes and start doing something approaching real history, you get introduced to “there isn’t one simple version, there are various sources or there may not even be a source, two people can interpret the same event in different ways”.

          But even reading things older than “the last five years” will give you an idea that opinions have not been set in stone, things change, Back Then views we now think good/bad were thought to be bad/good for perfectly honest reasons according to the values and beliefs of the time or place. This is why I’m always pushing “Fiction! Read more fiction, and more older fiction! Popular novels usually have the attitudes that the people of the time accepted uncritically and unthinkingly, the conventional wisdom, and this will open your eyes to the idea of ‘there isn’t one simple right answer’!”

          It’s probably my bias towards the arts and humanities peeking out here 🙂

        • Aegeus says:

          I don’t know what sort of high school you went to, but mine did teach how to evaluate a source for credibility and reliability, several times in several contexts. Research papers in history (along with an entire book on historiography). Persuasive and informative essays and debates in English class (including a section on propaganda techniques). The complaints about how students only learn to regurgitate what the teacher taught them just don’t match my experiences.

          But on the other hand, Julie K, above, sounds like she had to write just as many research papers, but somehow didn’t learn the underlying lesson. So what was the difference in how we were taught?

          • Who wouldn't want to be Anonymous says:

            That doesn’t match my experience at all. No reasons were given except “shut up and do it because it is part of the assignment.”

      • The crowdsourcing of consensus through social media is preferable to letting the mainstream media assume that role.

        • onyomi says:

          I think one of the biggest changes in my lifetime, in terms of reliability of knowledge, was when I realized that Wikipedia was more reliable than your average encyclopedia.

          • E. Harding says:

            Only on some articles, especially the most visited and contested ones with an even balance of opinion. Sometimes, when one side wins in a particularly bitter contest, the results can be quite ugly (e.g., “The Cyrus Cylinder bears striking similarities to older Mesopotamian royal inscriptions.” -what’s the word “striking” doing in an encyclopedia? and the Wikipedia articles on The Exodus and on Emma Sulkowicz, where in the latter case the Encyclopedia Dramatica article is more informative and reliable than the Wikipedia article).

          • onyomi says:

            But it’s still the best, on net, if not in every single case.

    • Richard says:

      How about “Those who participate in the study decide nothing. Those who report on the study decide everything.” ?