Nelson J.D., Meder B., Szalay C., Crupi V., and Tentori K., Implications of disregarding objective utilities when selecting a medical test, Annual Meeting of the Psychonomic Society (Long Beach, November 23, 2014).
Abstract. Consider the task of selecting a binary medical test to determine whether a patient has a disease. Normatively, this requires considering the base rate of the disease, the true and false positive rate for each test, and the payoffs and costs for correct and incorrect diagnoses. However, it can be difficult for people to appropriately use objective utilities. Can shortcut pure information strategies sometimes identify the objectively most useful test? We explore this through simulation studies and mathematical proofs. We prove that one simple strategy, the likelihood difference heuristic, identifies the objectively most useful test in some conditions. We also characterize the performance of information gain, probability gain, and related pure information test-selection strategies. If the objective payoff structure and base rate of a disease are even approximately known, it can be possible to identify a purely informational test selection strategy that performs well despite not explicitly calculating objective utilities.