Combining multiple methods – whether it is System 1 or 2 – is much more predictive than only using one method. Different methods often measure different variables, which explain only part of the puzzle, e.g. eye tracking measures attention, survey understanding, etc. Therefore, using multiple methods helps provide a bigger picture. But why is this so important?
Why Multiple Methods Make Sense? They Measure Different Variables
In the chart below, you will find an overview of methods measuring key performance factors for packaging. Eye tracking is the best for measuring visibility, survey for likeability, and virtual shopping for purchase intent. Though you can try to measure purchase intent with a 7-point Likert scale, that will only give you a 0.2 correlation with purchase behavior. Virtual shopping boasts a 0.8 correlation.
Adding Multiple Methods Creates a Bigger Picture
If package testing shows a high purchase intent and is not seen, or is highly visible, but people are not interested, then it will not sell. You need to have both high visibility and interest.
There is a correlation between the methods (high attention might indicate high interest), so their joint effect – 1+1 – is not 2, but rather 1.5. The table below shows the combined predictive value of different methods, assuming an overlap of 0.7.
What does that mean in practice? In validation studies, we noticed that surveys could correctly predict the success of 6/10 in a pool of good and bad ads. Eye tracking could vouch for 7/10, facial coding for 7/10, and virtual shopping for about 8/10. However, by combining these methods, we are able to select 9/10 good ads.
In a hypothetical situation, a 10mio ad budget might result in a 30mio sales uplift if the campaign is successful. In the worst-case scenario, there will be either no or negative impact. You can increase your chances with testing – using surveys, for instance, can lead to 18mio uplift, based on the survey’s predicting power. On the other hand, combining different methods may increase the uplift to 27mio!
| Method / predictive power | Correct prediction | Average sales uplift |
| Survey | 6/10 | 18mio |
| Eye tracking | 7/10 | 21mio |
| Facial coding | 7/10 | 21mio |
| Virtual shopping | 8/10 | 24mio |
| Combined | 9/10 | 27mio |
| Method | Predictive value | Description of method | Reasoning behind predictive value |
| Eye tracking | 2 | Measures the gaze direction to understand how many people saw the package, how long they are looking to the package design, etc. | Research has shown that a 10% increase in visibility will result in an increase in sales by 2%. So, we consider this twice more useful than survey alone. That’s why they get a factor 2 for predictive value. In eye tracking data, you will also find more data on differences between designs than in survey. |
| RTM | 1.4 | Measures the emotional certainty of attitudes | Measures the speed of implicit reactions and provides 40% higher accuracy than surveys That is why they get a factor 1.4. |
| Facial coding | 0.5 | Measures emotions through facial expressions | Facial coding is still very experimental in shopper research but has potential as shoppers only glance at products for a split second. Only RTM and facial coding can measure their spontaneous, initial reactions. |
| Traditional opinion surveys | 1 | Recall, purchase question, brand attributes | Asking a respondent to indicate from a scale from 1-7 whether he or she would buy the product, has a correlation of about 0.2 with real shopper behavior. |
| Virtual shopping | 2.4 | Measure purchase intent in a real-life shopping environment. | Virtual shopping data has a correlation of 0.6-0.95 with in-store purchase data. Given this much higher correlation, we give it a factor 2.4. |
This summary clearly shows that relying solely on questionnaires will not provide accurate data on the impact of your campaign. Combining complementary methods offers more reliable insights and consequently leads to better business decisions.
Disclaimer: The views and perspectives expressed in this article are those of the EyeSee team. Examples of analysis performed within this article are only examples.
Interested in learning more about combining explicit and implicit methods? Do not miss our article on “behavioral revolution” in the market research industry.