Did assessors select attributes by chance alone in your TDS study, and how relevant is it to know?
Dominance rates arising from Temporal Dominance of Sensations (TDS) data are almost always plotted and understood with reference to chance and significance lines, which are based on the assumption of random attribute selection. These lines are auxiliary to the chart, and used to interpret the dominance rates; when the dominance rate for some attribute exceeds the significance line, the conventional interpretation is that this attribute is being noticed significantly. Thus, these auxiliary lines are used to provide a convenient, qualitative interpretation of TDS data. However, the concept of the significance lines has some deficiencies that we explore in this paper.
We were interested in what real TDS data look like under a true null hypothesis of random attribute selection. To obtain such data, we sampled from real TDS data (keeping products and assessors’ selection patterns intact) to create dominance rates based on random attribute selections. Dominance rates exceed the significance line about 5% of the time (across attributes and time points), as expected. These curves differed from any real TDS curves that we have ever observed previously insofar as that they were unusually flattened. Next, we sampled again from real TDS data (keeping attributes and assessors’ selection patterns intact) to create dominance rates based on attribute selections for random (hybrid) products. Derived TDS curves much more closely resemble curves from real TDS studies and show a lot of “significances”, even though products are artificially created by the permutations. This is due to properties of the products that are inherent to the product category: e.g., hard will occur early during the evaluation of a chocolate (if at all), but not towards the end.
From these simulations, we reach two conclusions. First, we recognize that the hypothesis that assessors select the dominant attribute completely at random is almost always false. Second, the reason that it is false is that, regardless of the product, there is a time-dependent category signature present, as well as competition between attributes, such that significances for one attribute depend on which other attributes are on the list. These properties limit the informative value of the conventional TDS significance lines that are widely used. As an alternative, we explore the possibility of context-dependent significance thresholds which are customized for and aligned with research objectives. These reference lines are recommended for improved interpretation of data arising from TDS studies.
Meyners, M., & Castura, J.C. (2019). Did assessors select attributes by chance alone in your TDS study, and how relevant is it to know? Food Research International, 119, 571-583. https://doi.org/10.1016/j.foodres.2018.10.035