Statistical evaluation of panel repeatability in Check-All-That-Apply questions
Methodologies for evaluating panel repeatability in Check-All-That-Apply (CATA) questions are reviewed and developed. First, the limitations with using McNemar’s test as suggested elsewhere for the evaluation of repeatability are demonstrated through simple examples. Alternative approaches are then suggested and discussed.