Conversion optimization with A/B tests

I’ve been thinking a bit about conversions and A/B tests lately. I’ve recently worked with a client on helping them pick imagery for their new website. In the end, we ended up with two different hero images that we all seemed to agree aligned with the company’s values and what we think will speak to potential customers.

The decision was then to A/B tests the two images to see which one converts better. While this is great, I’m wondering how one could create a third option. In order to create an A/B test, you need to run an additional script on your site (so 50% sees images A and 50% sees image B). That script - like any script - adds additional loading time. Ideally I’d like to A/B test two images, but I’d also like to A/B test two versions - one with the script and one without the script.

To understand how much time scripts add to a loading time, try visiting USA Today’s new “European Union Expierence” (e.g. without scripts and tracking).

Booking.com A/B tests everything. If something cannot be A/B tested, Booking.com won’t do it. There’s more than 1,000 A/B tests running at any time.Living a Testing Culture

Please share:TwitterLinkedInFacebook
Get more writing like this

Sign up and get new writing, just like this, every other two weeks. Unsubscribe any time (I'm not a dickhead).

Books

User Experiences that Matter (2016)
Mastering Freelance (2017)

If You're Getting Started in UX

What's a 'User Experience' Anyways?
How Do You Learn UX?
Working as a UX Designer

Next Steps in UX

Working as a UX Lead
Defining a UX Strategy
Writing as Part of the UX Process

Thought-pieces

AI Ethics - A New Skill for UX-Designers
Designer Ethics & The Moral Implications of our Apps
The Future of the UX-Designer
Voice Input’s Effect on Social Norms

The Work We Do

Chasing Growth
New Tools Don’t Always Equal Productivity
Why Designers Need to Write
The Tools I Use to Run My Business

Featured Writing & Interviews Elsewhere

Q&A With Anton Sten, Author of User Experiences that Matter - Adobe
What the F*#!ck is a UX Designer anyway - Working not Working
It’s Time for a Code of Ethics for Designers - Medium Modus
The Art of Going Freelance - .Net Magazine
It Takes Time - Being Freelance episode 100

From My Newsletter

Who’s listening?

UX of Email Newsletters

Working as a UX-lead

2018 in review

What’s my location?

I’m taking a break

Stay humble, stay eager

Back to Work!

Vanity Metrics

The Future of Retail

2017 review

What do you do?

Carpe Diem UX-designers

What´s Good Design?

Chasing Growth

A Redesign

Is Less More?

Why Simple is Hard

Pricing It Perfectly

Trusting Your Gut

Built to last

An Eye on the Future

UX Design explained

Bite-sized Posts

The Enemy

The Hot Potato Process

Leave the Phone at Home

Delight Comes Last

The Cost of Lies

Big Mood Machine

Simplicity is a war

The next iPhone

Leadership or management

Everyone should own a dog

Silence is gold

Cameras that understand

Humans, not users

Keeping AI Honest

Right to privacy

The State of UX in 2019

Why scrap scrappy?

Organized for browsing

The iPhone Franchise

Why Small Teams Win

The Bullshit Web

Just keep at it

Let them eat cake

Netflix Culture

Skype

Unfoundered

Tech is not Neutral

Productivity

Whose risk?

Why Small Teams Win

Phone Bored

Karim Rashid

Dieter Rams

Bleeding Out

Fake News is spam

Conversational Design

Dropbox

Bye bye Facebook

Cuba

The seat at the table

Givenchy

Love letters to trees

Pricing Philisophy

Specialize

Personas

Make me think

Hawaii Missile Alert

Why Design Systems fail

What You Build

Checkout for Winners

Living a Testing Culture

Creative Class

How To Predict Your Future

Github

Medium

Design quotes

Enough

Why?

ARKit

Designing for Mobile

Failure, Reflect, Renew

Growth

Working with me

Great user experience

Naming your icons

Conversations

All writing

Who’s listening?

The Enemy

The Hot Potato Process

Leave the Phone at Home

UX of Email Newsletters

Delight Comes Last

The Cost of Lies

Big Mood Machine

Simplicity is a war

The next iPhone

Working as a UX-lead

Leadership or management

Everyone should own a dog

Silence is gold

Cameras that understand

Humans, not users

Keeping AI Honest

Right to privacy

2018 in review

What’s my location?

The State of UX in 2019

Why scrap scrappy?

I’m taking a break

Organized for browsing

Stay humble, stay eager

The iPhone Franchise

Why Small Teams Win

Back to Work!

The Bullshit Web

Just keep at it

Let them eat cake

Netflix Culture

Skype

Unfoundered

Vanity Metrics

Tech is not Neutral

Productivity

Whose risk?

Why Small Teams Win

Phone Bored

Karim Rashid

Dieter Rams

Bleeding Out

Fake News is spam

Conversational Design

Dropbox

Bye bye Facebook

Cuba

The seat at the table

Givenchy

Love letters to trees

Pricing Philisophy

Specialize

Personas

Make me think

The Future of Retail

Hawaii Missile Alert

2017 review

Why Design Systems fail

What You Build

Checkout for Winners

Living a Testing Culture

What do you do?

Creative Class

How To Predict Your Future

Carpe Diem UX-designers

What´s Good Design?

Chasing Growth

Github

Medium

A Redesign

Design quotes

Enough

Why?

ARKit

Designing for Mobile

Is Less More?

Why Simple is Hard

Pricing It Perfectly

Failure, Reflect, Renew

Trusting Your Gut

Built to last

An Eye on the Future

Growth

Working with me

UX Design explained

Great user experience

Naming your icons

Conversations