Testing the adequacy of formal models of an irrational learning effect

Abstract

The inverse base-rate effect (IBRE) is an irrational phenomenon in predictive learning characterized by as a preference for rare, unlikely outcomes in the face of ambiguity. This thesis investigates the adequacy of formal explanations for this puzzling phenomenon. In the first project, I will focus on mechanisms of learning that mathematical models posit underlie this preference. A class of attentional explanation produces a counter-intuitive prediction: the effect disappears under concurrent load. I confirm the prediction, but only when participants were under an obvious time constraint -- irrationality reduces under increased task demands. This suggests that multiple learning mechanisms operate independently and are differentially affected by concurrent load. In the second project, I test basic assumptions of the most prominent theories: this irrational bias depends on prediction error. Here, I gradually removed elements of a predictive learning design to test the extent to which error-driven processes underlie this bias. Throughout my attempts, the inverse base-rate effect persisted and remained robust. This outcome suggests that this irrational bias is independent of supervised learning procedures - a big change in the problem structures of the IBRE. In the third project, I look for the most adequate formal computational model of the canonical IBRE. In addition to group-level accommodation, I also incorporate heterogeneity into the benchmark. To accomplish this, I developed g-distance, which incorporates the extent to which models exhibit a similar range of behaviors to the humans they model. Applying it to five models of the IBRE reveals that none of the models outperform a random model. While analyzing the human data, I also discovered that the group-level result was observed in less than 1% of individuals. These projects provide new insight into the IBRE and how we should approach building and evaluating models of the IBRE and associated phenomena. I will discuss these insights in detail and how they influence future research on the IBRE.