The age-old problem of quality versus quantity always raises its head when planning user tests. Should you be testing a broad audience, or refining your market? Should users be moderated or unmoderated?
There are plenty of benefits in testing both quantitative data and qualitative data. The question you need to answer is, which form of data is more valuable to you as an organisation?
Getting the right data
Quantitative data offers value in numbers. Although it is considerably less detailed than qualitative data, it presents a completely objective outlook on user testing. A good example of this is if you were an e-commerce site and you wanted to determine if your call-to-action, say a ‘Buy Now’ button, was more effective placed on the right-hand side of the page or at the bottom of the page. You could run this test over the course of a week and then from the collected results determine which position would be more effective for your store.
Qualitative data is a detailed outlook on user experience. It’s asking why to every decision, process or thought process of a user. It is generally a longer task to execute in comparison to quantitative research but can provide you with valuable insights you would not have gained otherwise. Taking from the previous example, if you are an e-commerce site and want to know the thought processes of users leading up to your conversion page, you would run a moderated user test. In our experience, one-on-one user tests are more valuable compared to multiple users in one session.
To moderate or not to moderate
In relation to moderation, it’s generally best practice to have quantitative research unmoderated and qualitative research moderated. This is to ensure time is not spent where it’s not needed and you have the opportunity to personally interpret users reactions that are not encapsulated with words or notes.
This information should help you understand which data type has the most influence on the decisions in your organisation. User testing is a valuable investment into future strategies, as it allows you to look through the lens of a consumer and gain useful insights into how to improve your online results.