Replicated tetrad simulations for sensitivity quantification and panelist selection
Thurstonian-derived models are used widely for interpretation of sensory discrimination test results. Estimates of d´ are a signal-to-noise ratios, for which measurement sensitivity provides important context. Sensitivity is often defined descriptively (e.g. employees) rather than quantitatively (e.g. employees with sensitivity 1.2±0.2 for the relevant product category). We sought to select discrimination panelists based on quantified sensitivity estimates, and to investigate the effect of such selection on test power, panel homogeneity, and bias and variance of d´ estimates.
Simulation tests provide true values against which sensitivity and d´ estimates can be evaluated, permitting objective targets that cannot be obtained from real-world data, where true parameters remain unknown to the researcher. An extensive tetrad test simulation study was conducted. More than 18,600 simulated panels for various panel sizes, rep sizes, true product effects, and sensitivity distributions. In Stage 1, simulations were used to obtain panelist sensitivity estimates. In Stage 2, panelists were selected based on their estimated sensitivities. Evaluation data was simulated, yielding d´ estimates that could be compared with true values.
Simulation testing results illustrate the inverse relationship between the number of observations (panel size, number of reps) and bias and variance of d´ estimates, and the positive relationship with power. Panel heterogeneity was low, and trivial for panel sizes and replicates above 16 and 3, respectively. Marginal estimates seem a more practical choice than subject-specific estimates for practitioners because they are faster and more reliable to obtain, and perform similarly with regards to bias and variance of d´ estimates.
Castura, J. C., Ennis, J. M., & Christensen, R. H. B. (2014). Replicated tetrad simulations for sensitivity quantification and panelist selection. 12th Sensometrics Meeting. 29 July – 1 August. Chicago, IL, USA.