Existing and new approaches for analysing data from Check All That Apply questions
Check-All-That-Apply (CATA) questions are increasingly being incorporated into consumer tests because they provide a simple mechanism for consumers to communicate their perceptions of products being evaluated. We review existing and propose new approaches for analysing data obtained from such a study.
Contingency tables are well known, and can be pictured using mosaic plots. Correspondence analysis (CA) using the Χ2 distance provides dimensionality reduction, but Hellinger’s distance is often preferred where rarely cited attributes skew results. Word clouds can be used to determine citation frequency for responses that might be entered in open comment format by consumers (e.g. upon checking “other” in a CATA question). Cochran’s Q test provides a univariate test for differences between 3 or more products, and the sign test can be used to assess pairwise differences. To our knowledge no omnibus hypothesis test is available for assessing global differences. We propose such a test, based on randomization and Cochran’s Q statistics, in which the null distribution is formed from data re-randomizations. Multidimensional alignment (MDA) is suggested to investigate the relationship between products and CATA attributes. The ϕ-coefficients, proposed to understand relationships between CATA attributes, are readily visualized using MDS. Consumers can be asked to evaluate an ideal product, and the gaps between the real and ideal products can inform product improvements. Penalty and penalty-lift analyses can reveal (positive and negative) hedonic drivers.
Methods are illustrated by means of CATA study on whole grain breads.
Meyners, M., Carr, B.T., & Castura, J. C. (2013). Existing and new approaches for analysing data from Check All That Apply questions. 10th Pangborn Sensory Science Symposium. 11-15 August. Rio de Janeiro, Brazil.