Measuring the impact of Design

Over the last several years, we’ve seen a massive shift in the design world. While we once evaluated our work based on outputs — how effective a design artifact appeared to be — we have recently begun to focus more on whether a design helps drive business results. In other words, design used to be all about aesthetics — does it look cool? Today, it’s all about outcomes.    

“In the early days of UX design and digital design, there was a notion of design being the pixels; the final layer after the strategy had been plotted out and structured and planned,” says our Director of Product Design Sarah McIlwain. “You’d give that plan to your design team, and they would make it look nice. There was a model like a short-order cook, basically.” 

Of course, things move fast in the tech world, so inevitably, design had to keep up. As UX design and design research matured, methods like Agile and Lean developed. 

“There became this idea of measuring the success of something by the impact it has on the customer,” says McIlwain. “In other words, ‘If we do this right, what gets better in someone's life?’”

Tying design to outcomes is becoming the norm

As the perspective has shifted to be outcome-focused, many businesses are putting more pressure on design leaders to prove the value of their team’s work. In order to pivot to this way of thinking, design leaders have had to rethink the way they measure success. 

Especially today, most design leaders are responsible for measuring and reporting on the outcomes of design for their team. Some leaders save this for performance review time or as a part of the promotion process, but others say that’s not enough.

Encouraging your team to think often about the impact of their contribution—not just what they do, but the effect their work has—helps product designers bring that perspective to every project they do, big and small. This makes it much easier to connect your work to a business metric or outcome. 

This way of thinking is becoming more widespread. Based on user research that we conducted in the State of Design 2021, we found that respondents reported:

The challenge of measuring design outcomes

Tying design to business outcomes is as important as it is hard to nail down.

“One of the challenges with leading with design is that the attributes that make a good designer, a good strategist, also make a good head of product, and a good head of growth. They're just things that make you good at business,” says Melissa Cullens, former CXO of Ellevest and founder and CEO of Charette. 

But while you can tie hard metrics to growth and product efficacy, it’s much harder to do so with design. 

“There are so many arguments about how we measure design,” says Andy Vitale, VP of Product Design and Content at Quicken Loans. “Of course we need to continue to move the needle and show outcomes, but it's hard to measure every little thing.”

This seems to be where things start to fall apart for today’s design teams. It’s clear what they want to measure: the impact of design on the end-user. But there’s no one direct way to get there. 

In other words, there are both qualitative and quantitative measures that teams can look at to understand the impact that design has on the business, but each has its limitations. 

Measuring qualitative design results

McIlwain believes that while growth metrics are important for measuring success, they’re not the only thing you should be looking at.

“I think the key to design being able to measure success is being in conversation with your product leaders; having a very close relationship with product [teams] and being able to work with them to understand, ‘What is the customer problem we're trying to solve?’” 

In this situation, it’s often a change in customer behavior that signals success. Design leaders need to say, ‘What is the behavior we want to see if we do this right?’ and then use that as a measuring stick. Hopefully, this change in behavior can be quantified and measured. “A lot of that is just done through user research and helping us understand that we're on the right track,” says Vitale.

Ultimately, if designers are clear about what they’re trying to achieve with their work from the outset, they’ll have something to measure. “As long as designers can clearly articulate and communicate the intended outcome and the results that we achieved… design is the scientific method. Put out a hypothesis, see if it's right, be completely okay with being proven wrong and creating a better iteration the next time around,” says Vitale.

Measuring quantitative design results 

While it’s still very difficult to draw a straight line from design outcomes to business outcomes, design leaders are finding more creative ways every day to try

One approach is to coach designers to connect their work to a business metric or business outcome. By committing to this way of thinking, you can find new ways to quantify the unquantifiable. It can take a bit of creativity to get to a place where you can actually quantify your impact, but it’s rarely impossible.

For example, suppose a design team wanted to measure the effectiveness of their investment of energy into cross-functional relationships. In that case, they might look at velocity of decision-making as a success metric. 

Often, finding a way to measure something just takes a bit of brainstorming. Don’t shy away from sitting down with your team and spending some time asking, “What are all the ways we could quantify this relationship by measuring it?”

What needs to happen for measurement to improve? 

Measuring the impact of design is not impossible, but it takes a lot of work, creativity, and inference to get there. Will there ever be a way to tie design success directly to business success? Maybe not. But it’s possible the answer lies, yet again, in a perspective shift. We need to accept the idea that while some things can be quantified, others will forever elude quantifiable measurement.

“Think about Shopify’s checkout flow,” says Cullens. “It’s amazing. They have got it down to this art, and they didn’t get there through qualitative testing. They got there through machine learning and hundreds of people flowing through the same basic action of making a purchase. There isn't anybody on a team that can make design decisions better than a computer in that moment.”

She adds that fighting over that kind of plumbing is unnecessary. There is some quantifiable information you can only get from automation, from data and insights. Once teams accept that, they can put their design brains somewhere else.

In other words, perhaps quantitative measures are best left to quantifiable outcomes in order to free up more space for qualitative exploration. Qualitative research often leads to unexpected discoveries about what you could be doing that you’re not already, says Cullens. “If we could focus our teams more on really setting some ships up to sail and seeing which ones go, that seems like a really good place to put a human.”

Want to read more about the challenges today’s designers are facing—and how to solve them? Read the State of Design 2021 report.