Measuring UX with HEART
August 26, 2019 in Product design
One of the most important steps in developing any digital project is defining what success means is. Because we have the possibility to measure virtually everything, it’s easy to get caught in the trap of measuring all the wrong things. We often throw these numbers around and, of course, the bigger the number is, the more impressive it is. We tend to do this with almost everything nowadays! Page views, email subscribers, instagram followers, funding raised, revenue, and even numbers of employees. Big is better, right? Don't get me wrong, in some cases big is better - like profit, but in some cases it's just vanity metrics.
The only metrics that entrepreneurs should invest energy in collecting are those that help them make decisions. Unfortunately, the majority of data available in off-the-shelf analytics packages are what I call Vanity Metrics. They might make you feel good, but they don’t offer clear guidance for what to do.
When you hear companies doing PR about the billions of messages sent using their product, or the total GDP of their economy, think vanity metrics. But there are examples closer to home. Consider the most basic of all reports: the total number of “hits” to your website. Let’s say you have 10,000. Now what? Do you really know what actions you took in the past that drove those visitors to you, and do you really know which actions to take next? In most cases, I don’t think it’s very helpful.Eric Ries on Tim Ferriss
With all of the tools available showing us all the charts, how do we know which are useful and which are just vanity metrics? Hint: the answer is right there in the question — "u-s-e-f-u-l." Metrics that help us make decisions are useful, metrics that can't be used to make actionable steps are vanity metrics.
It all comes down to one thing: does the metric help you make decisions? When you see the metric, do you know what you need to do? If you don’t, you’re probably looking at a vanity metric. Vanity metrics are all those data points that make us feel good if they go up but don’t help us make decisions.Neil Patel - Metrics, Metrics On The Wall, Who’s The Vainest Of Them All?
Measuring UX
While other areas have it easier to measure their progress in numbers, UX is still tougher to tackle. Because UX is so difficult to measure, we still struggle to define our value to organizations who could benefit from our work.
Business metrics are often tied directly to dollars and cents, so it's a metric almost everyone understands. Improved sales and reduced costs equal profit and everyone loves money. If it's a business that relies on technology they already have tons of stuff they measure. Reduced server loads, faster software, and less latency are all things that while not everyone in the public truly understand them, most understand them as a good thing.
But UX, and design in general, is harder to tackle. One way to measure its effect is through the HEART framework, designed by Kerry Rodden, Hilary Hutchinson, and Xin Fu from Google’s research team.
To make this work in practice it’s important to use the right metrics. Basic traffic metrics (like overall page views or number of unique users) are easy to track and give a good baseline on how your site is doing, but they are often not very useful for evaluating the impact of UX changes. This is because they are very general, and usually don’t relate directly to either the quality of the user experience or the goals of your project — it’s hard to make them actionable.How to choose the right UX metrics for your product
The framework is a kind of UX metrics scorecard that’s broken down into 5 factors:
- Happiness: How do users feel about your product? Happiness is typically measured by user satisfaction surveys, app ratings and reviews, and net promoter score.
- Engagement: How often are people coming back to use the product? Level of user involvement, typically measured via behavioral proxies such as frequency, intensity, or depth of interaction over some time period. Examples might include the number of visits per user per week or the number of photos uploaded per user per day.
- Adoption: How many people successfully complete the on-boarding process and become regular users? Adoption is measured by number of new users over a period of time or percentage of customers using a new feature.
- Retention: The rate at which existing users are returning. You can measure how many of the active users from a given time period are still present in some later time period. As a product owner, you may be more interested in failure to retain, commonly known as “churn.”
- Task success: Can your users achieve their goal or task quickly and easily? Task success is measured by factors like efficiency (how long it takes users to complete the task), effectiveness (percent of tasks completed), and error rate.
Using the HEART framework gives you a better understanding of the user and their relationship with the product. The nice perk of this is that it can be applied to a single feature in your app or to your whole product. It gives you the option to measure just what you need to at any moment.
Informed by data, driven by empathy
The other day, I published this quote from Data-Driven Design is Killing Our Instincts that's stuck with me.
We’re told all design decisions must be validated by user feedback or business success metrics. Analytics are measuring the design effectiveness of every tweak and change we make. If it can’t be proven to work in a prototype, A/B test, or MVP, it’s not worth trying at all.
In this cutthroat world of data-driven design, we’re starting to lose sight of something we once cherished: the designer’s instinct. “Trusting your gut” now means “lazy, entitled designer.” When we can ask users what they want directly, there’s no room for instinct and guesswork.
Or is there?
As we have access to all of this data, and the general consensus is that data should lead the way, I'm confused by what the future role of a designer is. What I truly like about the HEART framework is that while the output of it is informed data, it's all done through the eye and mind of a human.
Just like we need to be careful to not measure everything just because we can, we need to remember that in order to really understand human's behaviors, we need to talk with them, not just look at charts. Charts are great for an overview, but human insight is the best way to understand the underlying reason for the chart's direction.
“Data-driven” is all the rage at the moment, everyone wants a slice of the “big data” cake. Data scientists are the new rock stars, replacing the JavaScript and Front-end gurus and ninjas from a few years back. My problem with trends like these is that they cause the so called “tunnel-vision”. Thing x is a trend right now and we should do that too because,… you know… everyone’s doing it.
Some companies have already started to realise that data alone can’t answer all the questions they need answered. Booking.com put “Informed by data, driven by empathy” in their design guidelines.Measuring and Quantifying User Experience
How do you get from the HEART categories to metrics you can actually implement and track? Unfortunately, there’s no off-the-shelf “HEART dashboard” that will magically do this for you. The most useful metrics are likely to be specific to your particular product or project. Personally, I like to start bigger projects with a UX Strategy. This UX Strategy should define what the most important tasks and metrics for your product is, and consequentially, what failure looks like. How might success or failure in the goals actually manifest itself in user behavior or attitudes? For YouTube, an engagement signal might be the number of videos users watch, but an even better one could be the amount of time they spend watching those videos. A failure category for YouTube Search might be entering a query, but not clicking on any of the results.
Even fashion brands are using data to design products that will meet customers demands better:
“You never design by data, but the data provides a compass as you’re navigating a hunch.”Analytics are reshaping fashion's old-school instincts
So while we as designers might not get the out-of-the-box dashboard with all the charts ready for us to act on, there are tools and techniques that we can use to get the information needed for us to act upon to get a confident start. Let me know if you need help getting started!
More Articles in Product design
You need to confirm your email to confirm your subscription.
-
March, 2024
-
February, 2024
-
January, 2024
-
December, 2023
-
November, 2023
-
October, 2023
-
September, 2023
-
August, 2023
-
July, 2023
-
June, 2023
-
May, 2023
-
April, 2023
-
March, 2023
-
January, 2023
-
December, 2022
-
November, 2022
-
September, 2022
-
August, 2022
-
July, 2022
-
June, 2022
-
May, 2022
-
April, 2022
-
January, 2022
-
December, 2021
-
November, 2021
-
October, 2021
-
September, 2021
-
August, 2021
-
July, 2021
-
June, 2021
-
May, 2021
-
April, 2021
-
March, 2021
-
January, 2021
-
November, 2020
-
October, 2020
-
September, 2020
-
August, 2020
-
June, 2020
-
May, 2020
-
April, 2020
-
March, 2020
- Regarding Zoom
- Why wireframes are becoming obsolete
- Get static
- Emotions and work - know yourself
- All books are now free
- Wireframes are too concrete
- Stuck at home
- What Great Managers Do
- From Error to Understanding: The Importance of Brand Voice When Things Go Wrong
- Answering your question - How I design
- The Sliding Scale of Giving a Fuck
- Hard startups
-
February, 2020
-
January, 2020
- Debbie Millman - Time Well Spent
- The last tracker was just removed from Basecamp.com
- Fathom Analytics
- Redesign to learn
- How putting a price on speed helped one retailer increase mobile revenue in just 2 months
- Accessibility drives aesthetics
- Smart people use simple words to make others feel smart.
- The great paradox of our time: everything is both better and worse than ever before
-
December, 2019
-
November, 2019
-
October, 2019
-
September, 2019
-
August, 2019
- How we work with microcopy
- Measuring UX with HEART
- Mobile E-Commerce UX: Deemphasize ‘Install App’ Ads or Avoid Them Entirely
- Analytics are reshaping fashion’s old-school instincts
- What to Do When You Feel Uninspired at Work
- Stay in your lane and outlast your competition
- A Mindful Approach to Technology
- “I Just Knew I Was Going to Surpass These Guys I Was Working For”
- People Are Starting to Realize How Voice Assistants Actually Work
- Fast Software, the Best Software
- Data-Driven Design Is Killing Our Instincts
- Freelance and Business And Stuff
- UX can’t be defined by one set of "rules"
-
July, 2019
- 5 Years - 300+ Posts!
- Superhuman and email privacy
- Why Consultants Quit Their Jobs to Go Independent
- Give them candy: small details make all the difference
- Your inbox is spying on you
- On Modus: It’s Time for a Code of Ethics for Designers
- Frank Chimero on causing ’good trouble’ and re-imagining the status quo to combat achievement culture
- Advertising: contextual vs. behavioral and why it matters
-
June, 2019
- Citrix improves UX and cuts costs by 65%
- Delight Comes Last
- Tim Cook’s Stanford Commencement Address
- You care more about your privacy than you think
- The Cost of Lies
- Let’s assemble like the Avengers and... do work
- Big Mood Machine
- Simplicity is a war
- How Notion Is Going After Atlassian and Why It Just Might Win
- Hiring a Management Consultancy for Digital Is a Mistake
- Just write: why Product Designers should write, and how to get started
- Dos and dont’s on designing for accessibility
- I’ve redesigned my website and it looks exactly the same
-
May, 2019
-
April, 2019
- Scapegoating User Experience Design
- Clip sharing with Overcast
- Why don’t we just use Material Design?
- Accenture sued $32m+ over website redesign for Hertz
- Venture capital money kills more businesses than it helps
- Creating a UX Strategy
- The elegance of nothing
- Amazon Workers Are Listening to What You Tell Alexa
- Working as a UX-lead
- Gumroadday: Get my books at -17% discount
-
March, 2019
- Working as a UX designer
- Meet Q: The First Genderless Voice
- It Doesn’t Have to Be Crazy at Work
- How Notion pulled itself back from the brink of failure
- What’s the meaning of work?
- Wireframes are becoming less relevant — and that’s a good thing
- Neurodiversity and the Digital Divide: how our neurological differences shape the way we experience the web
- Accessibility, a powerful design tool
-
February, 2019
-
January, 2019
-
December, 2018
-
November, 2018
-
October, 2018
-
September, 2018
-
August, 2018
- Living among baby robots
- Yuval Noah Harari on what the year 2050 has in store for humankind
- Voice Input’s Effect on Social Norms
- How I kicked my email compulsion
- Facebook's struggle to moderate itself
- The Secret to Ant Efficiency Is Idleness
- Google Employees Are Organizing To Protest the Company’s Secret Search Engine
- Why Small Teams Win
- Back to Work!
- The Bullshit Web
- Nikhil Sonnad on the banal evil of Facebook
-
July, 2018
- Five questions UX employers should be asking
- Just keep at it
- Yes, Alan, There Is An ROI For UX Design
- High Fives, AI, and Connecting the Dots: MailChimp’s VP of Design on What Business can Learn from Design
- Rands Information Practises
- The Race to a Trillion
- What Elon Musk Should Learn From the Thailand Cave Rescue
- Cheap Shower Curtains
- Tech Sector Values are Broken?
-
June, 2018
- Confirmation Bias in Design
- Rise of the machines: has technology evolved beyond our control?
- Good product writing is conversational
- Retail is Not Dying, But it Has to Evolve
- The Netflix Binge Factory
- We are all trapped in the “Feed”
- Stop “feeding” your users
- Turn off your push notifications. All of them.
- From the Drawing Board to the Board Room
- Think inclusively at every step
- No classes, no professors: the alternative to business school
- Designer Ethics & The Moral Implications of our Apps
- Conversion optimization with A/B tests
-
May, 2018
- AI Ethics - A New Skill for UX-Designers
- Netlify now shows your deploy status on its favicon
- Let them eat cake
- Netflix Culture
- Skype
- How do you learn UX?
- The New Google Assistant
- What Happened to Apple’s Whimsy?
- Who would you trust more with your data, Apple or the government?
- Unfoundered
- Microcontent: A Few Small Words Have a Mega Impact on Business
- Twitter logged all user’s passwords in plain text
- Requests for Personal Data From Apple, Google, and Facebook Compared
- What’s in a pattern name
- GDPR is ruining my life
-
April, 2018
- A Modest Guide to Productivity
- Google Debuts a Standalone To-Do App, Google Tasks
- Apple Should Make an Instagram Clone
- Vanity Metrics
- Tech is not Neutral
- Facebook to exclude North American users from some privacy enhancements
- Netflix has to "produce great content, market it well, serve it up beautifully."
- Sen. Blumenthal: “Your business model is to maximize profit over privacy.”
- Productivity
- My #1 Piece of Advice to Freelancers: Lean Into The Moment
- Design Systems: Better UX through Defined Standards
- Tim Cook - We care about the user experience
- Europe’s tough new data-protection law
- The Real Technology Problem
- Why Small Teams Win
- Whose risk?
- Phone Bored
- Karim Rashid
- Creative Class is open
- Dieter Rams
- Being cash-free puts us at risk of attack: Swedes turn against cashlessness
- What’s a ‘User Experience’ Anyways?
-
March, 2018
- Fake News is spam
- Bleeding Out
- How Tech Giants Design For Transgender Users–Or Don’t
- Don't use the F word (freelancer)
- Dropbox
- Conversational Design
- Everyone Is Going Through Something
- It Doesn’t Have to Be Crazy at Work
- Flight Crews Don’t Decide Where To Fly
- New Tools Don’t Always Equal Productivity
- The seat at the table
- Why Can Everyone Spot Fake News But YouTube, Facebook And Google?
- Cuba
- Bye bye Facebook
- Why I hate your fake redesign
- Givenchy
- Diversity in the design industry
- Your competitors don’t matter
- Apple’s "Sound on the Go" Strategy
- The Most Important Design Skill For An AI-Dominated World
- Love letters to trees
- Amazon has a fix for Alexa’s creepy laughs
- You have to make sure that you’re focused on the thing that matters.
- Voice Input is the Next Big Thing - Or is It?
-
February, 2018
- Specialize
- Pricing Philisophy
- AMP for email is a terrible idea
- Is this Finnish school the perfect design?
- The #1 reason Facebook won’t ever change
- A Year of Learning and Leading UX at Google
- C.A.R.E - A simple framework for user onboarding
- Inside Facebook´s Hellish Two Years
- A better user experience...or?
- Personas
- Reflections on working on an iPad
- Everything Easy is Hard Again
- How to succeed at freelancing
- The iPhone X is Apple’s underrated masterpiece
- Make me think
-
January, 2018
-
December, 2017
-
November, 2017
-
October, 2017
-
September, 2017
-
August, 2017
-
July, 2017
-
June, 2017
- The Why before the Why
- No Share Buttons on Mobile Sites
- Groundwork for Creating Great User Experiences
- ARKit
- 4 Reasons Why Your Body Text Should be Bigger
- Now that Apple is talking about AR and VR, should we be too?
- Think like an artist to create better designs
- Investing in businesses and products
- 10 Reasons Why All Designers Should Start Writing More
- Become a better UX-designer with these three words
- Vox Media Accessibility Guidelines
- Writing for UX: some practical tips
-
May, 2017
-
April, 2017
-
March, 2017
-
February, 2017
-
January, 2017
-
December, 2016
-
November, 2016
-
October, 2016
-
September, 2016
-
August, 2016
-
July, 2016
-
June, 2016
-
May, 2016
-
April, 2016
-
March, 2016
-
February, 2016
-
January, 2016
-
December, 2015
-
November, 2015
-
October, 2015
-
September, 2015
-
August, 2015
-
July, 2015
-
June, 2015
-
May, 2015
-
April, 2015
-
November, 2014
-
April, 2013