Do panellists donkey vote in sensory Choose-All-That-Apply questions?
A so-called donkey voter selects candidates according to position on an election ballot. Are untrained sensory panellists similarly influenced by position when responding to choose-all-that-apply (CATA) questions? In sensory and consumer testing, lists of choices, conventionally presented in fixed order, allow panellists to indicate sensory perceptions without requirements for scaling. Results help in understanding products and drivers of hedonic response.
Using Compusense at-hand, colleagues at University of Arkansas and Compusense Inc. presented 10 commercial orange juices to 106 student panellists. Separate CATA questions were presented for different sensory modalities as follows: 5 appearance choices (one column), 28 flavour choices (3 columns of 10, 10, and 8), and 9 texture choices (1 column). In each case “none of these apply” appeared in the final position (quite different and rarely selected, it was omitted from analysis), and during sessions 3 and 4 all other choices were presented using a Williams design with choice sets assigned to sample sets. The Next button appeared at bottom right.
First positions increased selection percentage points for appearance (+5.9%), flavour (+2.6%) and texture (+2.8%). Attributes in the leftmost flavour column were chosen more than either those in middle (+2.5%) or rightmost (+3.6%) columns. First position added 10-20% selections by proportion. Various data adjustments were considered to confirm the absence of artifacts. Computerized visualizations were developed to vividly demonstrate results. Results raise strong concerns that fixed choice order ballots skew CATA results, with implications for anyone conducting sensory and consumer tests in this manner. Rotation of samples is commonplace in designed experiments, and rotation of choices, as performed in this study, is recommended for improving data quality.
Castura, J. C. (2009). Do panellists donkey vote in sensory choose-all-that-apply questions? 8th Pangborn Sensory Science Symposium. 26-30 July. Florence, Italy. (Oral).