Refinement of a statistical framework for an own-brand large-scale consumer quality program
Retailers benefit from understanding how consumers perceive the quality and value of their own-brand offerings, but a systematic approach is necessary with many products. A well designed consumer testing program provides efficiency and structure for ongoing quality monitoring, as well as guidance for product reformulation. Decision trees provide classifications of products relative to competitors, and inform decisions regarding reformulation. We seek to refine an existing statistical framework in which the two-tailed paired t-test is used to analyze consumer data from tests with crossover designs. Three statistical tests were investigated. The paired t-test assumes an underlying normal distribution. The Wilcoxon signed-rank test assumes that data are symmetrically distributed around the median. The Sign test makes no assumptions regarding symmetry about the median. Simulations conducted by Li examine statistical power when data are generated from underlying Normal, Uniform, and Cauchy distributions with various parameterizations. These simulations informed development of a new decision tree, which was then evaluated using real data collected in 406 consumer tests (each n>80, in which consumers overall opinion was expressed on a 9-point scale). Significant skewness (p<0.05) was detected in 47 of the 406 data sets. Simulation indicated the suitability of the Wilcoxon signed-rank test. In 2 of the 47 cases the decision made by the Wilcoxon signed-rank test differed from the decision obtained from the paired t-test. When skewness is not significant simulations indicated the suitability of the paired t-test when sample distributions are light-tailed, and the Wilcoxon signed-rank when sample distributions are heavy-tailed. Of the 359 data sets that were not significantly skewed, the conclusion differed on only 6 light-tailed and 6 heavy-tailed data sets. Taking the lower p-value provides a reasonable heuristic for practitioners. Investigations underscore the effectiveness of the existing framework, and also suggest simple refinements that deliver improved guidance for some data sets.
Li, M., Castura, J. C., McNicholas, P., & Hasted, A. (2011). Refinement of a statistical framework for an own-brand large-scale consumer quality program. 9th Pangborn Sensory Science Symposium. 4-8 September. Toronto, ON, Canada.