Distribution assumptions underlying consumer testing
To evaluate in-market products, routine sensory quality tests are conducted using verified consumers. Each product is matched to a competitor product with similar attributes, and presented to consumers by experimental design. We seek to refine a statistical framework in which consumer opinions are analyzed with 2 objectives: to provide an actionable decision regarding reformulation, and to provide classifications of products as being above, below, or not different than competitors. An actionable non-inferiority test (one-side, with appropriate alpha level) leads to product reformulation, while classification is a two-tailed test. Three statistical tests were investigated. The paired t-test assumes an underlying normal distribution. The Wilcoxon signed-rank test assumes that data are symmetrically distributed around the median. The sign test makes no assumptions regarding symmetry about the median. Statistical analyses were conducted on data sets from 468 consumer tests, each with complete data for n>80 consumer respondents, with overall opinion evaluated on a 9-point category liking scale. The Shapiro-Wilk hypothesis test, which is rejected when evidence indicates that data do not come from an underlying normal distribution, indicated a non-normal distribution for 288 of the 468 data sets at the 5% level. The paired t-test and the Wilcoxon test hypothesis test gave different conclusions regarding significance for 7 of these data sets, and data were grossly skewed for 90 of the data sets.
Li, M., Castura, J. C., & McNicholas, P. D. (2011). Distribution assumptions underlying consumer testing. Institute of Food Technologists (IFT) 2011 Annual Meeting + Food Expo. 11-14 June. New Orleans, LA, USA.