Do panelists “monkey-pick” attributes in TDS studies and how relevant is it to know?
Dominance rates arising from temporal dominance of sensations (TDS) data are almost always plotted and understood with reference to chance and significance lines. Chance lines are fully determined by the number of attributes, and represent what we might expect if poor reading but task-engaged monkeys picked the attributes. Significance lines are conventionally based on the 95% upper confidence limit for chance, therefore relating to a null hypothesis of attribute monkey-picking.
To investigate the relevance of the hypothesis, we sampled from real TDS data to create putatively monkey-picked dominance rates. Dominance rates exceed the significance line about 5% of the time (across attributes and time points), as expected. It is also found that these curves look quite unusual. However, with any statistical test, we want to differentiate products that have some property from those that do not. If the test were relevant, we should also observe real curves like the resampled ones, but we do not. Why?
We argue that this is due to the hypothesis of monkey-picking being known a priori to be false. To demonstrate, we sampled again from the TDS data, this time across products but keeping the attribute associations intact. Derived TDS curves closely resemble those from real studies, and show a lot of “significances”, despite the fact that the data are random. This is due to properties of the products that are inherent to the category: e.g., hard and bitter will occur early during the evaluation of a chocolate (if at all), but not towards the end.
In addition to temporal aspects, there is also competition between attributes, such that significances for one attribute depend on which other attributes are on the list. As an alternative, we explore the possibility of context-dependent auxiliary lines which are customized for and aligned with research objectives.
Meyners, M., & Castura, J. C. (2017). Do panelists “monkey-pick” attributes in TDS studies and how relevant is it to know? 12th Pangborn Sensory Science Symposium. 20-24 August. Providence, RI, USA.