Confirmation Bias in Design

June 25, 2018

I’ve recently been reading a book called The Black Swan by Nassim Nicholas Taleb and it got me thinking about our confirmed way of thinking. The book focuses on the extreme impact of certain kinds of rare and unpredictable events and how we strive to find simplistic explanations for these events later on. The problem, as Nassim explains, is that we place odds on past events repeating themselves even if they are rare, unpredictable, and - most of all - unrepeatable.

He calls these events Black Swans, a reference to a 17th century philosophical thought experiment. In Europe, anyone could have told you the ‘fact’ that “all swans are white” since that is all they’d seen. So what was the chance of seeing a black one? Zero? Just like imagining a green swan today, seeing black swans were beyond rare events until 1697 when explorers found black swans in Australia.

Nassim argues that most of the really big events in our world are rare and unpredictable, and thus trying to extract generalizable stories to explain them may be emotionally satisfying, but it's practically useless. September 11th is one such example, and stock market crashes are another. Or, as he puts it, "History does not crawl, it jumps."

Four hundred years ago, Francis Bacon warned that our minds are wired to deceive us. "Beware the fallacies into which undisciplined thinkers most easily fall--they are the real distorting prisms of human nature." Chief among them: "Assuming more order than exists in chaotic nature." Now consider the typical stock market report: "Today investors bid shares down out of concern over Iranian oil production." Sigh. We're still doing it.

Forecasting is not just at the heart of Wall Street, but it’s something each of us does every time we make an insurance payment or strap on a seat belt.Chris Anderson

I’m fascinated by this line of thought and try to view it through the lens of my daily work. What black swans are hidden in the work we do? Perhaps because of the fact that digital products have evolved so quickly, we are even more prone to accept ‘facts’ that are in front of us. You and I need to accept our lives are loaded with confirmation biases.

People generally see what they look for and hear what they listen for.Harper Lee, To Kill a Mockingbird

Confirmation bias is everywhere

Consider how common smoking in public places like airplanes and restaurants was just 20-30 years ago. It’s not that strange considering in the 18th and early 19th centuries, tobacco smoke was considered good for your heart and lungs. All of tobacco’s health ‘benefits’ were explored and some were outright strange. Tobacco enemas were so popular that they were placed along the banks of the river Thames to help drowning victims! This is a perfect example of confirmation bias at work.

Confirmation bias - our tendency to accept evidence we agree with at face value and dismiss information we don't agree with unless the evidence is overwhelming. Confirmation biases limits our ability to seek out and uncover the truth.

I see this over and over in my own work and in other’s everyday. We run A/B tests until they show the impact that we were hoping for and stop there. Even when the tests aren’t showing us the result we were hoping for, we make up stories for the failure and why our original theory still stands true. Self-validation. Confirmation bias.

Don’t build solutions in search of a problem. Just because you had an idea doesn’t mean someone needs it. The mere existence of a solution doesn’t validate the existence of a problem.
Josh Pigford

Michael Aagaard talks about this in great detail and suggest that, rather than testing our ideas, we should do detective work to find out what our customers really need and how they talk about it. We can use this new information to A/B test against our previous data and see success. This may change everything about how we develop products.

Overcoming confirmation bias

This is something I’ve experienced many times. Here are a few things I try to keep in the front of my mind:

  1. People generally don’t know what they want. They know what they have and what they don’t like about it. Finding the solution to their problem is your work.

  2. People usually say one thing, but act very differently. This is why the best user research is multi-faceted: talking to users, observing users, analyzing statistics, and sometimes trying crazy things. User action is much more relevant than user feedback. Feedback from users is great for understanding their line of thinking, but not great for understanding their actions. Words are their idea of what they do, actions are the reality. It’s not uncommon that these two don’t align.

  3. Statistics are a great tool, but always remember that statistics are only relevant to the actions currently available. You can only draw insights from what you currently have in that moment. It’s not a great metric for looking at what people want and certainly can’t predict the future.

Our brains are wired for narrative, not statistical uncertainty. And so we tell ourselves simple stories to explain complex thing we don't--and, most importantly, can't--know. The truth is that we have no idea why stock markets go up or down on any given day, and whatever reason we give is sure to be grossly simplified, if not flat out wrong.Chris Anderson

How to overcome confirmation bias? Like most things in life, it starts with accepting the fact that you could be wrong. Seek out a different perspective. Why? Because being biased towards information that confirms what we already believe often leads to errors in judgment and costly mistakes in marketing, something I’m guessing none of us wants.

I’ve previously written of how I think more UX-designers - and people in general - should be more comfortable with the words ‘I don’t know’. Accepting that there is so much in this world that we simply cannot explain much less replicate is key. Nassim argues that this is why one shouldn’t read the news. The more we think we know, the less likely we are to be open to unpredictable events. Reading the news simply gives us an idea of what has happened and it’s in some extent naive of us to assume that this is related to what will happen.

To sum up, confirmation bias is a very real part of our lives, both work and personal. Through skills like observation, communication, and analysis of the information in front of us, we can begin to sort out our personal “black swans”.

It’s honest, authentic, and accessible.

I love sharing my experiences working in design and what’ve I’ve learned along the way. Join a community of thousands of designers, developers, and product professionals by signing up to my newsletter!

Not quite ready to sign up? I totally understand! Why not start by reading some of my past issues?

Great! Just “one more thing”...

You need to confirm your email to confirm your subscription.

  1. February, 2024

  2. January, 2024

  3. December, 2023

  4. November, 2023

  5. October, 2023

  6. September, 2023

  7. August, 2023

  8. July, 2023

  9. June, 2023

  10. May, 2023

  11. April, 2023

  12. March, 2023

  13. January, 2023

  14. December, 2022

  15. November, 2022

  16. September, 2022

  17. August, 2022

  18. July, 2022

  19. June, 2022

  20. May, 2022

  21. April, 2022

  22. January, 2022

  23. December, 2021

  24. November, 2021

  25. October, 2021

  26. September, 2021

  27. August, 2021

  28. July, 2021

  29. June, 2021

  30. May, 2021

  31. April, 2021

  32. March, 2021

  33. January, 2021

  34. November, 2020

  35. October, 2020

  36. September, 2020

  37. August, 2020

  38. June, 2020

  39. May, 2020

  40. April, 2020

  41. March, 2020

  42. February, 2020

  43. January, 2020

  44. December, 2019

  45. November, 2019

  46. October, 2019

  47. September, 2019

  48. August, 2019

  49. July, 2019

  50. June, 2019

  51. May, 2019

  52. April, 2019

  53. March, 2019

  54. February, 2019

  55. January, 2019

  56. December, 2018

  57. November, 2018

  58. October, 2018

  59. September, 2018

  60. August, 2018

  61. July, 2018

  62. June, 2018

  63. May, 2018

  64. April, 2018

  65. March, 2018

  66. February, 2018

  67. January, 2018

  68. December, 2017

  69. November, 2017

  70. October, 2017

  71. September, 2017

  72. August, 2017

  73. July, 2017

  74. June, 2017

  75. May, 2017

  76. April, 2017

  77. March, 2017

  78. February, 2017

  79. January, 2017

  80. December, 2016

  81. November, 2016

  82. October, 2016

  83. September, 2016

  84. August, 2016

  85. July, 2016

  86. June, 2016

  87. May, 2016

  88. April, 2016

  89. March, 2016

  90. February, 2016

  91. January, 2016

  92. December, 2015

  93. November, 2015

  94. October, 2015

  95. September, 2015

  96. August, 2015

  97. July, 2015

  98. June, 2015

  99. May, 2015

  100. April, 2015

  101. November, 2014

  102. April, 2013