Confirmation Bias in Design

I’ve recently been reading a book called The Black Swan by Nassim Nicholas Taleb and it got me thinking about our confirmed way of thinking. The book focuses on the extreme impact of certain kinds of rare and unpredictable events and how we strive to find simplistic explanations for these events later on. The problem, as Nassim explains, is that we place odds on past events repeating themselves even if they are rare, unpredictable, and - most of all - unrepeatable.

He calls these events Black Swans, a reference to a 17th century philosophical thought experiment. In Europe, anyone could have told you the ‘fact’ that “all swans are white” since that is all they’d seen. So what was the chance of seeing a black one? Zero? Just like imagining a green swan today, seeing black swans were beyond rare events until 1697 when explorers found black swans in Australia.

Nassim argues that most of the really big events in our world are rare and unpredictable, and thus trying to extract generalizable stories to explain them may be emotionally satisfying, but it’s practically useless. September 11th is one such example, and stock market crashes are another. Or, as he puts it, “History does not crawl, it jumps.”

Four hundred years ago, Francis Bacon warned that our minds are wired to deceive us. “Beware the fallacies into which undisciplined thinkers most easily fall—they are the real distorting prisms of human nature.” Chief among them: “Assuming more order than exists in chaotic nature.” Now consider the typical stock market report: “Today investors bid shares down out of concern over Iranian oil production.” Sigh. We’re still doing it.

Forecasting is not just at the heart of Wall Street, but it’s something each of us does every time we make an insurance payment or strap on a seat belt.Chris Anderson

I’m fascinated by this line of thought and try to view it through the lens of my daily work. What black swans are hidden in the work we do? Perhaps because of the fact that digital products have evolved so quickly, we are even more prone to accept ‘facts’ that are in front of us. You and I need to accept our lives are loaded with confirmation biases.

People generally see what they look for and hear what they listen for.Harper Lee, To Kill a Mockingbird

Confirmation bias is everywhere

Consider how common smoking in public places like airplanes and restaurants was just 20-30 years ago. It’s not that strange considering in the 18th and early 19th centuries, tobacco smoke was considered good for your heart and lungs. All of tobacco’s health ‘benefits’ were explored and some were outright strange. Tobacco enemas were so popular that they were placed along the banks of the river Thames to help drowning victims! This is a perfect example of confirmation bias at work.

Confirmation bias - our tendency to accept evidence we agree with at face value and dismiss information we don’t agree with unless the evidence is overwhelming. Confirmation biases limits our ability to seek out and uncover the truth.

I see this over and over in my own work and in other’s everyday. We run A/B tests until they show the impact that we were hoping for and stop there. Even when the tests aren’t showing us the result we were hoping for, we make up stories for the failure and why our original theory still stands true. Self-validation. Confirmation bias.

Don’t build solutions in search of a problem. Just because you had an idea doesn’t mean someone needs it. The mere existence of a solution doesn’t validate the existence of a problem. Josh Pigford

Michael Aagaard talks about this in great detail and suggest that, rather than testing our ideas, we should do detective work to find out what our customers really need and how they talk about it. We can use this new information to A/B test against our previous data and see success. This may change everything about how we develop products.

Overcoming confirmation bias

This is something I’ve experienced many times. Here are a few things I try to keep in the front of my mind:

  1. People generally don’t know what they want. They know what they have and what they don’t like about it. Finding the solution to their problem is your work.

  2. People usually say one thing, but act very differently. This is why the best user research is multi-faceted: talking to users, observing users, analyzing statistics, and sometimes trying crazy things. User action is much more relevant than user feedback. Feedback from users is great for understanding their line of thinking, but not great for understanding their actions. Words are their idea of what they do, actions are the reality. It’s not uncommon that these two don’t align.

  3. Statistics are a great tool, but always remember that statistics are only relevant to the actions currently available. You can only draw insights from what you currently have in that moment. It’s not a great metric for looking at what people want and certainly can’t predict the future.

Our brains are wired for narrative, not statistical uncertainty. And so we tell ourselves simple stories to explain complex thing we don’t—and, most importantly, can’t—know. The truth is that we have no idea why stock markets go up or down on any given day, and whatever reason we give is sure to be grossly simplified, if not flat out wrong.Chris Anderson

How to overcome confirmation bias? Like most things in life, it starts with accepting the fact that you could be wrong. Seek out a different perspective. Why? Because being biased towards information that confirms what we already believe often leads to errors in judgment and costly mistakes in marketing, something I’m guessing none of us wants.

I’ve previously written of how I think more UX-designers - and people in general - should be more comfortable with the words ‘I don’t know’. Accepting that there is so much in this world that we simply cannot explain much less replicate is key. Nassim argues that this is why one shouldn’t read the news. The more we think we know, the less likely we are to be open to unpredictable events. Reading the news simply gives us an idea of what has happened and it’s in some extent naive of us to assume that this is related to what will happen.

To sum up, confirmation bias is a very real part of our lives, both work and personal. Through skills like observation, communication, and analysis of the information in front of us, we can begin to sort out our personal “black swans”.

Please share:TwitterLinkedInFacebook
Get more writing like this

Sign up and get new writing, just like this, every other two weeks. Unsubscribe any time (I'm not a dickhead).

Books

User Experiences that Matter (2016)
Mastering Freelance (2017)

If You're Getting Started in UX

What's a 'User Experience' Anyways?
How Do You Learn UX?
Working as a UX Designer

Next Steps in UX

Working as a UX Lead
Defining a UX Strategy
Writing as Part of the UX Process

Thought-pieces

AI Ethics - A New Skill for UX-Designers
Designer Ethics & The Moral Implications of our Apps
The Future of the UX-Designer
Voice Input’s Effect on Social Norms

The Work We Do

Chasing Growth
New Tools Don’t Always Equal Productivity
Why Designers Need to Write
The Tools I Use to Run My Business

Featured Writing & Interviews Elsewhere

Q&A With Anton Sten, Author of User Experiences that Matter - Adobe
What the F*#!ck is a UX Designer anyway - Working not Working
It’s Time for a Code of Ethics for Designers - Medium Modus
The Art of Going Freelance - .Net Magazine
It Takes Time - Being Freelance episode 100

From My Newsletter

UX of Email Newsletters

Working as a UX-lead

2018 in review

What’s my location?

I’m taking a break

Stay humble, stay eager

Back to Work!

Vanity Metrics

The Future of Retail

2017 review

What do you do?

Carpe Diem UX-designers

What´s Good Design?

Chasing Growth

A Redesign

Is Less More?

Why Simple is Hard

Pricing It Perfectly

Trusting Your Gut

Built to last

An Eye on the Future

UX Design explained

Bite-sized Posts

Leave the Phone at Home

Delight Comes Last

The Cost of Lies

Big Mood Machine

Simplicity is a war

The next iPhone

Leadership or management

Everyone should own a dog

Silence is gold

Cameras that understand

Humans, not users

Keeping AI Honest

Right to privacy

The State of UX in 2019

Why scrap scrappy?

Organized for browsing

The iPhone Franchise

Why Small Teams Win

The Bullshit Web

Just keep at it

Let them eat cake

Netflix Culture

Skype

Unfoundered

Tech is not Neutral

Productivity

Whose risk?

Why Small Teams Win

Phone Bored

Karim Rashid

Dieter Rams

Bleeding Out

Fake News is spam

Conversational Design

Dropbox

Bye bye Facebook

Cuba

The seat at the table

Givenchy

Love letters to trees

Pricing Philisophy

Specialize

Personas

Make me think

Hawaii Missile Alert

Why Design Systems fail

What You Build

Checkout for Winners

Living a Testing Culture

Creative Class

How To Predict Your Future

Github

Medium

Design quotes

Enough

Why?

ARKit

Designing for Mobile

Failure, Reflect, Renew

Growth

Working with me

Great user experience

Naming your icons

Conversations

All writing

Leave the Phone at Home

UX of Email Newsletters

Delight Comes Last

The Cost of Lies

Big Mood Machine

Simplicity is a war

The next iPhone

Working as a UX-lead

Leadership or management

Everyone should own a dog

Silence is gold

Cameras that understand

Humans, not users

Keeping AI Honest

Right to privacy

2018 in review

What’s my location?

The State of UX in 2019

Why scrap scrappy?

I’m taking a break

Organized for browsing

Stay humble, stay eager

The iPhone Franchise

Why Small Teams Win

Back to Work!

The Bullshit Web

Just keep at it

Let them eat cake

Netflix Culture

Skype

Unfoundered

Vanity Metrics

Tech is not Neutral

Productivity

Whose risk?

Why Small Teams Win

Phone Bored

Karim Rashid

Dieter Rams

Bleeding Out

Fake News is spam

Conversational Design

Dropbox

Bye bye Facebook

Cuba

The seat at the table

Givenchy

Love letters to trees

Pricing Philisophy

Specialize

Personas

Make me think

The Future of Retail

Hawaii Missile Alert

2017 review

Why Design Systems fail

What You Build

Checkout for Winners

Living a Testing Culture

What do you do?

Creative Class

How To Predict Your Future

Carpe Diem UX-designers

What´s Good Design?

Chasing Growth

Github

Medium

A Redesign

Design quotes

Enough

Why?

ARKit

Designing for Mobile

Is Less More?

Why Simple is Hard

Pricing It Perfectly

Failure, Reflect, Renew

Trusting Your Gut

Built to last

An Eye on the Future

Growth

Working with me

UX Design explained

Great user experience

Naming your icons

Conversations