Today, mastering mobile UX is crucial to succeed in the digital landscape because it strongly impacts customer loyalty, reduces costs, and increases sales. The challenge remains to understand how users feel and interact in the mobile space… Few solutions exist for behavioral data collection that quantify mobile user experience, let alone transforming that data into the kind of insights that can help conquer the mobile channel.
EyeSee developed a market research lab in the cloud that offers at least 3 complementary solutions to test mobile UX, each with its own advantages:
1) Live website testing on mobile devices (Qualitative)
2) Testing mock-ups on mobile devices (Behavioral & Quantitative)
3) Emulation of the mobile content on a computer screen (Behavioral & Quantitative)
Solution 1: Mobile testing on live websites
Testing live websites on mobile devices only appears to be the best solution, when in reality it suffers from a couple key shortcomings due to sample sizes and current phone technology.
The trouble with sampling arises from respondents’ need to download an app to their device that records their screen, voice and/or webcam data. Unfortunately, few consumers are willing to do so.
The problem that surfaces because of current phone technology is that when respondents enable all the features necessary to record the screen, taps, sounds, etc. battery life is very limited. There is thus a risk of the phone dying in the middle of the test. A likely related issue is that the phone heats up, causing respondents to be distracted from the test.
Because of these two disadvantages, the live website testing solution is best suited for projects that only require small sample sizes. Still, the future is very promising as technology and social norms will evolve and slowly eliminate these sampling and tech issues.
Solution 2: Mobile mock-up testing
Testing mobile website mock-ups on a mobile device solves the sampling and technological issues. Respondents do not need to download an app, which opens the door to the recruitment of large samples. The approach works as follows: EyeSee creates a mock-up website that closely resembles a real online shop, like Amazon.com for example. The mock-up is then hosted on EyeSee servers and posted to the web via a hidden link. Test participants are then directed to EyeSee’s page and asked to complete several tasks.
This solution automatically records participants’ mobile browsing activity through the mock-up website. Respondents can thus browse the mock-up with their mobile devices as they normally would in real life. The battery and heat-related technological issues of solution 1 are circumvented because no extra recording features need to be activated on the mobile devices. Thanks to these advantages, this solution a great fit for projects that require large sample sizes.
Respondents are given multiple tasks (e.g. “purchase shampoo”) that allow us to track:
- Whether they can complete the tasks
- What is on the screen
- How they navigate to complete the task (clicks/taps/swipes).
After performing the tasks, respondents fill out a survey.
This solution provides useful, hard facts about what respondents do on the mobile site. The drawback, though, is that it does not answer why respondents do what they do. To overcome this limitation, it is possible to combine solutions 1 and 2.
Still, both solutions 1 and 2 still face a common constraint: because they are performed on mobile devices, respondents have less patience when completing tasks – this means that tests must be kept as short as possible. Solution 3 addresses that constraint and has some other very important benefits
Solution 3: Emulation on desktop
This solution uses respondents’ PCs to display the mobile website inside a frame that mimics a real mobile device. Despite the counter-intuitive setup, the approach provides the most advantages in terms of panel sizes and use of advanced research methods.
Recruiting a panel of respondents for PC-based testing is a more mature process than for mobile-based testing. Potential respondents are less apprehensive about completing tests on their computers compared to on their mobile devices. As a result, recruiting samples of more than 100 respondents – even for specific target groups in other countries – is much easier. In addition, all methods included in solutions 1 and 2 (think aloud, completion rate, navigation, viewable…) can be combined with:
- Eye tracking: tracks people’s eyes to know where they are looking at
- Facial coding: measures emotions through facial expressions
- As well as longer, more in-depth questionnaires
The best solution for mobile UX insights is a combination of multiple methods
A single silver bullet solution for testing mobile users’ experience doesn’t exist yet. However, it is possible to gain effective actionable insights for mobile by combining different testing solutions:
- Testing on live websites and Mock-up testing get the most possible information from actual use of the real mobile device.
- Mock-up testing and Desktop Emulation track behavior (navigation, completion rate, screen taping) on a large sample size, ensuring reliability and validity of the data.
- Testing on live websites and Desktop Emulation provide the opportunity to conduct think aloud protocols that capture respondents’ thoughts.
- Desktop Emulation provides the most introspective data by allowing for longer surveys, eye tracking and facial coding.