Analysis And A/B Tests Of UX

People do A/B testing in the most of the cases, but do not notice it. I've already created 7 versions of Fland-lab website and I think this is a hard mode of A/B testing. Since, I have done it manually and without any testing tools.

The most useful tool that was in my hands is Google Analytics. Someone told me to create an account there and set up my website to understand the behavior of my visitors even before I started work on the very first version of the website.

I've learned different metric values and parameters to understand what is actually happening on my website and what can I do to improve interaction. This approach has helped me, partly, but not so much. Since, all this time the most attractive section that I had on the website was a portfolio. And I didn't sell anything or promote.

Analytics feature works for me, but this is not No 1 instrument for me. So, the A/B testing as well. The best way to improve something on this kind of websites is to get a feedback from real people or even better from clients or other specialists in your area.

However, it's useful for eCommerce and Web service projects where I worked on improvements as UX designer. So, before making any decision about improvements it's make sense to learn behaviors of users from traffic analytics tools such as Google Analytics or Yandex Metrika. We'll talk about analytics and A/B testing tools and methods in this article to understand the basics and start using them to improve the value of your work.

Analytics tools and methods

I'm not professional analytic and I'll not load you with lots of professional knowledge here, but I'll tell what I used in my experience and what works for me in cases of improvements of User Interface and sometimes User Experience at all.

My 1st priority section in Google Analytics is behavior. As a UX designer for me is important to see all the scenarios of how people use websites. What are the most heat places and what places are cold.

Usually after that I take a look at heat map and behavior map or how I call it user flows. There I can see the interaction of users and the website. It helps me to understand where I loose visitors and what is their interaction patterns.

Sometimes I use Yandex Metrika to see the broader picture of using the website. It allows to watch a video on how people interact with web interface or specific parts of the interface.

As I mentioned before in some of the cases we should tend to require feedback about the website directly on a meeting with customers or on a specific interview with the target audience. Of course analytics tools automates this process and give you statistics in a way of charts and tables. But personal requests and interviews will give you a more direct response and more specific data for analysis.

A/B testing tools and methods

What is important for A/B testing is that we need to launch different versions of one interface, page, functional at a time, but not one after another. This will give you more accurate result. And usually it's called a split-test.

As for tools I know about Visual Website Optimizer and KISS metrics as alternative ways to GA. Since they have a better interface and the most common integrations (Google Analytics, Wordpress, e.g.). But why do not just use Google Analytics? For instance, VWO shows that a new version B is better because of the specific reason and it allows you to split traffic for different versions.

Google Analytics works on the idea of a session. A person bounces around on a website and leaves, which is considered a "session", but if they return, that is a new session. Tools like VWO uses unique visitors and no matter how many visits they count as one visitor. This is critically important as 10 people and 10 visits by 1 person are not the same.

Thus, you can think that GA data are look like a mess. And you need to do a lot of custom work to make it work.

Of course we can build our own software for our specific needs. This solution is good, but expensive in a sense of time and money.

What about methods? Let's start with types of metrics and after that I'll show you an example how we can use one of A/B testing methods.

Some time ago Eric Ries wrote an article about Vanity Metrics and Actionable Metrics. Which is good from a practical point of view. Since, Vanity Metrics are not making sense of a real practice.

We have a number of Actionable Metrics:

Split-tests - based on comparing 2 versions of anything from minor copy tweaks to major changes in the product or its positioning.

Per-customer metrics - focuses our attention on real customers. For instance, instead of looking at the total number of page views in a given month, consider looking at the number of page views per new and returning customer.

Funnel metrics and cohort analysis - helpful for projects that have a couple of key customer lifecycle events: registering for the product, signing up for the free trial, using the product, and becoming a paying customer. I'll describe this type on example a little bit later in this article.

Keyword (SEM/SEO) metrics - allows to test different users, who were acquired with a given keyword as a segment and then track their metrics over time.

But let's talk about example to be more practical.

A good place to start with split-testing is to try moving UI elements around. For instance, you can rearrange the steps of your registration process for new customers.

Try to build funnel from analysis of interaction of users with 2 versions that you tested. You can have as a result two column table where each column is version of the page. And each row is a step of the interaction.

So, for example, you can have 100 registered users in both versions of the page. 50 of them are signed up in the first version and 30 users in the version 2. As a result, in version 1 you can have 20 sales of the product, but in the 2nd version only 5 cells. And you can see what option works better for you.

Final thoughts

You don't need to be sure on the 100% that Analytics data or A/B test results will give you what people want. You still can form desires of your users, surprise them and create something that they have never seen before in a good sense. But I recommend you use these analysis tools and methods as an insurance and better understanding of users. So, don't overwhelm yourself and make wise decisions.


Join my personal email list and receive an email from me every 1-2 weeks.

So if you like the stuff that I write on the website then leave your email to get it right into your inbox. No BS, spam or tricks... just useful content: