The highest predictive value of insight: Which methodology works best?

Combining multiple methods – whether it is System 1 or 2 – is much more predictive than only using one method. Different methods often measure different variables, which explain only part of the puzzle, e.g. eye tracking measures attention, survey understanding, etc. Therefore, using multiple methods helps provide a bigger picture. But why is this so important?

Why Multiple Methods Make Sense? They Measure Different Variables

In the chart below, you will find an overview of methods measuring key performance factors for packaging. Eye tracking is the best for measuring visibility, survey for likeability, and virtual shopping for purchase intent. Though you can try to measure purchase intent with a 7-point Likert scale, that will only give you a 0.2 correlation with purchase behavior. Virtual shopping boasts a 0.8 correlation.

Adding Multiple Methods Creates a Bigger Picture

If package testing shows a high purchase intent and is not seen, or is highly visible, but people are not interested, then it will not sell. You need to have both high visibility and interest.

There is a correlation between the methods (high attention might indicate high interest), so their joint effect – 1+1 – is not 2, but rather 1.5. The table below shows the combined predictive value of different methods, assuming an overlap of 0.7.

What does that mean in practice? In validation studies, we noticed that surveys could correctly predict the success of 6/10 in a pool of good and bad ads. Eye tracking could vouch for 7/10, facial coding for 7/10, and virtual shopping for about 8/10. However, by combining these methods, we are able to select 9/10 good ads.

In a hypothetical situation, a 10mio ad budget might result in a 30mio sales uplift if the campaign is successful. In the worst-case scenario, there will be either no or negative impact. You can increase your chances with testing – using surveys, for instance, can lead to 18mio uplift, based on the survey’s predicting power. On the other hand, combining different methods may increase the uplift to 27mio!

Method / predictive powerCorrect predictionAverage sales upliftSurvey6/1018mioEye tracking7/1021mioFacial coding7/1021mioVirtual shopping8/1024mioCombined9/1027mio

MethodPredictive valueDescription of methodReasoning behind predictive valueEye tracking2Measures the gaze direction to understand how many people saw the package, how long they are looking to the package design, etc.Research has shown that a 10% increase in visibility will result in an increase in sales by 2%. So, we consider this twice more useful than survey alone. That’s why they get a factor 2 for predictive value. In eye tracking data, you will also find more data on differences between designs than in survey.RTM1.4Measures the emotional certainty of attitudesMeasures the speed of implicit reactions and provides 40% higher accuracy than surveys That is why they get a factor 1.4.Facial coding0.5Measures emotions through facial expressionsFacial coding is still very experimental in shopper research but has potential as shoppers only glance at products for a split second. Only RTM and facial coding can measure their spontaneous, initial reactions.Traditional opinion surveys1Recall, purchase question, brand attributesAsking a respondent to indicate from a scale from 1-7 whether he or she would buy the product, has a correlation of about 0.2 with real shopper behavior.Virtual shopping2.4Measure purchase intent in a real-life shopping environment.Virtual shopping data has a correlation of 0.6-0.95 with in-store purchase data. Given this much higher correlation, we give it a factor 2.4.

This summary clearly shows that relying solely on questionnaires will not provide accurate data on the impact of your campaign. Combining complementary methods offers more reliable insights and consequently leads to better business decisions.

Disclaimer: The views and perspectives expressed in this article are those of the EyeSee team. Examples of analysis performed within this article are only examples.

Interested in learning more about combining explicit and implicit methods? Do not miss our article on “behavioral revolution” in the market research industry.