A
AI feels like it’s been unleashed on us, particularly in the past year. It’s impacting lots of different industries but we haven’t felt the impact in user experience design – yet.
After testing ChatGPT-4 for UX audits of 12 different web pages, Baymard found the AI had an 80% false-positive error rate and a 20% accuracy rate in the UX suggestions it makes.
Although ChatGPT correctly identified some UX issues, it overlooked many others. Some of its suggestions were either harmful to the UX or a waste of time compared to the work of experienced UX researchers.
Those headline findings show that when it comes to UX, it’s still very much a human-in-the-loop process. There’s some early promise, and judging by AI’s rapid progress in other fields, we have to assume it’s going to get better.
At Each&Other, we love trialling new things and we’re always looking out for more efficient and effective ways to deliver world-class UX. Over the past 18 months, we’ve spent a lot of time investigating AI and how it will impact not just our industry but the sectors our clients operate in.
I’ll declare my own bias here: I’m a technology optimist. I’ve always been a futurist; I tend to think further down the road than is good for me! I find AI amazing and terrifying in equal measure, but I think it will be a game changer in our industry – and I believe its effects will mostly be positive.
Will we eventually get to a place where the human doesn’t need to be in the loop? Probably. Will there be so much data that you need humans in the loop? Also probably.
In the short term, I see some areas where AI could be incredibly powerful. Figma is the main wireframing tool we use at Each&Other, and we’re starting to see it automating tasks like padding and spacing.
Figma’s auto layout feature is one of their most robust examples of automation.
Another area automation can help is content creation as we build out websites. Often, the content from wireframes can end up in production. In projects I’m working on, I want people to think about the content that goes into fields on a page, or on a mobile interface.
We’re interested in it because in Each&Other’s UX research we prefer to do one-to-one user testing interviews between a facilitator and a participant, asking about that person’s experience of the digital interface. However, we’re constrained in the number of interviews we can do in a project. Typically, we do six or 12. Once – and this was an edge case – we did 32 interviews but it was very apparent after the first five conversations that the same themes were coming up again and again.
To be clear, we’re not exploring tools like Outset because we want to make the research quicker but because we want it to have more reach, and therefore more validity. Now, we could run 200 tests in a morning using the tool, whereas if we were to run 200 face-to-face interviews, those qualitative tests would take weeks.
AI can give everyone involved in a project an excellent head start on where to start improving the UX.
Another area where AI can address major pain points for a brand is the ability to parse text for keywords and themes. Today, many of them are unaware of the frustrations their users feel, unless they’re spotting patterns in the data or they’re listening carefully to their customers.
How powerful could it be to analyse key words for positive or negative sentiments?
Imagine gauging sentiment in the feedback that comes in on your website, potentially identifying issues far earlier in the process than before, so you can step in and address them before they blow up.
How powerful would it be if brands could analyse key words for positive or negative sentiment, and identify changes in website conversion rates and understand if they’re within norms or if there are patterns developing they need to worry about.
Usually the marketing channel is tasked with keeping an ear open, and there are tools that will highlight when a brand or product is mentioned. My take on this is, that should ideally be part of the product team’s remit, not marketing. Instead of adding an extra layer of people to digest the feedback and make a change request, the product team could harness AI to gauge feedback, and respond by incorporating that into the product.
Previously, you had to have lots of expertise to delve into this kind of data; you might have needed a web analytics expert to spot trends like people spending less time on a page, and tell whether that means they’re getting through a purchase faster or they’re abandoning a cart because the process is too difficult.
We trialled it on one project and found we didn’t need a second person taking notes, so it turned what would have been a two-person project over one week into work for 1 person over 6-7 days. That’s a 30% saving which is powerful.
The tool analyses the data at scale, which informs the necessary qualitative digging. To come back to my earlier point, there’s still a human in the loop who’s overseeing and checking the output.
It’s not flawless, but it does the job quite well. I can see a lot of these tools will creep into the workflow and will make steady gains, making this necessary work incrementally less tedious. And the thing with these tools is, they keep getting better, whether they’re using OpenAI or another LLM model under the hood.
We’re at the cusp of having AI tools that bring real business intelligence into product development in real time. And when you get real-time intelligence, it makes you truly agile and responsive.
And while there’s a stampede to adopt AI in some sectors, it’s worth stating that you can’t just deploy a tool and expect savings or benefits right away. One of the best things you can do now to get ready for AI is to think practically about how your business will interact with it today. You might get things done quicker, but as a business, does quicker suit you? Are your teams and processes today set up to react to real-time insight, while also dealing with business as usual?
For me, that’s the key takeaway: you need to be ready for the time that AI will give back to you.