Simplistic solutions to complex problems turns behavioural science into a dangerous pseudoscience (part 1): Confirmation bias and complexity

This three-part series critiques the research of discredited high-profile Cornell University food researcher Brian Wansink from a complexity point of view.

On the 20th of September, many news outlets carried headlines announcing that high-profile Cornell University food researcher Brian Wansink would be retiring, after a university committee found that he had committed academic misconduct (an example of one of these headlines is shown in Figure 1). This followed an announcement¹ the previous day that JAMA Network academic journals would be retracting six studies that included Wansink as an author, bringing his total number of retractions to 13.

Wansink is a behavioural scientist, and head of the Cornell University Food & Brand Lab. His research has linked eating habits to food environments, concluding that changes in those environments is a solution to chronic health issues including obesity and diabetes. An example of a food environment change is to serve meals on smaller plates, as shown in the Food & Brand Lab image above.

Cornell University Provost Michael I. Kotlikoff says in a statement that:

The committee found that Professor Wansink committed academic misconduct in his research and scholarship, including misreporting of research data, problematic statistical techniques, failure to properly document and preserve research results, and inappropriate authorship. As provided in Cornell policy, these findings were thoroughly reviewed by and upheld by Cornell’s dean of the faculty.

This is a significant fall from grace for someone who has been an academic celebrity. As well as being a prolific author of academic papers, Wansink has written best-selling books, appeared in numerous television programs, and influenced changes to food package sizes (Figure 2).

Confirmation bias and complexity

So how did this happen? Some of the media articles that announced the Wansink misconduct finding point to the problem issues in academic publishing that we’ve previously raised in our quality of science and science communication series. For example, in an article in NPR, Brett Dahlberg talks about how Wansink engaged in p-hacking, a widespread practice where researchers inappropriately manipulate their data to make their results look more statistically significant. In Mother Jones, Kiera Butler also raises the issue of p-hacking, and she further alerts to the tendency for journals to want to publish novel or “sexy” findings, when in reality results that are both captivating and scientifically sound are a rarity.

But other articles point to a deeper issue. Writing in The Atlantic, James Hamblin relates that:

The Wansink saga has forced reflection on my own lack of skepticism toward research that confirms what I already believe, in this case that food environments shape our eating behaviors. For example, among his other retracted studies are those finding that we buy more groceries when we shop hungry and order healthier food when we preorder lunch. All of this seems intuitive.

Writing in TreeHugger, which had actively supported and promoted Wansink’s research findings, Lloyd Alter similarly laments that:

I personally am conflicted and troubled. I liked Wansink’s work because it confirmed and reinforced my opinions and biases, many of which I had before I read his books and studies; I first wrote about small fridges before he even got his job at Cornell. Questioning his results makes me question my own.

What both James Hamblin and Lloyd Alter are alerting to is confirmation bias. A type of cognitive bias, confirmation bias “is the tendency to search for, interpret, favor, and recall information in a way that confirms one’s preexisting beliefs or hypotheses”.

The effect of this confirmation bias in behavioural science is that both scientists and the consumers of science end up beguiled by simplistic solutions that ignore the complex reality of problems. I’ve already looked at this in regard to another aspect of behavioral science – ‘nudge theory’.

In June 2017, I wrote an article critical of ‘nudge theory’ and its application, in response to Ebert & Freibichler’s paper proposing ‘nudge management’ as the new scientific management approach². Nudge theory rose to prominence with the publication of the 2008 book Nudge: Improving Decisions About Health, Wealth, and Happiness:

Nudge theory (or nudge) is a concept in behavioural science, political theory and economics which proposes positive reinforcement and indirect suggestions to try to achieve non-forced compliance to influence the motives, incentives and decision making of groups and individuals.

Echoing Wansink, Ebert & Freibichler put forward the way that Google lays out healthy food choices in its cafeteria as a classic example of the successful application of nudge management, arguing that this food environment change nudges employees towards improved health and therefore better productivity.

However, one of the criticisms I raised in my June 2017 article is that nudge theory ignores the full range of determinants of behaviour³, with the food environment being only just a part of the complex spectrum of issues that influences healthy eating. For example, in regard to obesity, one of the chronic health issues that Wansink has sought to address through food environment changes, the US Centers for Disease Control and Prevention (CDC) advises that:

Obesity is a complex health issue to address. Obesity results from a combination of causes and contributing factors, including individual factors such as behavior and genetics. Behaviors can include dietary patterns, physical activity, inactivity, medication use, and other exposures. Additional contributing factors in our society include the food and physical activity environment, education and skills, and food marketing and promotion.

You can immerse yourself in this complexity by exploring the interactive Obesity System Influence Diagram (Figure 3) and reading the associated 2007 UK Government Office of Science Foresight Program report Tackling Obesities: Future Choices – Obesity System Atlas⁴.

In response to my June 2017 article, a commenter stated that “I’d refer you to works of Brian Wansink on eating as a good evidence base for nudge theory working in real life solutions.” No, sadly, the works of Brian Wansink on eating are actually further evidence for why we need to be skeptical about nudge theory or any other theory or so-called solution that addresses just one aspect of any highly complex problem or issue. For this reason, I recently used concerns in regard to Wansink’s work as an example of why we need to be able to effectively judge the trustworthiness of studies as part of evidence-based knowledge management.

The complexity of chronic health issues such as obesity means that food environment changes will at best only nudge some people towards a healthier life. I say “at best” because the complex way in which all of the factors interact makes it very difficult for researchers to be able to definitively conclude that changes in one part of the system created an observed result.

Because of this, I contend that the only way that Wansink could get his simplistic food environment changes to appear to be effective solutions to complex health problems was through p-hacking and the other inappropriate practices identified by the Cornell committee. So, in this case, these practices are actually symptoms of the true underlying problem, rather than the problem in themselves.

Having said that, I don’t think that Wansink set out to intentionally do the wrong thing, but rather that he fell foul of continually reinforced confirmation bias, leading to a perpetual ignorance of the true complexity of the chronic health issues he was seeking to address. As we’ve seen in the quotes from James Hamblin in The Atlantic and Lloyd Alter in TreeHugger, Wansink certainly isn’t alone in this regard.

But, as innocent as it may be, advancing simplistic solutions in ignorance of complexity and compounding this ignorance through confirmation bias turns behavioural science into a pseudoscience. It’s also dangerous because of the negative consequences that result from simplistic solutions to the complex chronic health issue of obesity. I’ll be addressing these issues in parts 2 and 3 of this three-part series.

Next part (part 2): Why it’s pseudoscience.

Header image source: Small Plates Lose Weight is licensed under CC BY-NC-ND 4.0.

References: