An Axiomatization of Learning Rules when Counterfactuals are not Observed

In this paper we study learning procedures when counterfactuals (payoffs of not chosen actions) are not observed. The decision maker reasons in two steps: First, she updates her propensities for choosing each action after every payoff experience, where propensities can be interpreted as preferences. Then, she transforms these propensities into choice probabilities. We introduce a set of axioms on how propensities are updated and on how these propensities are translated into choices and study the decision marker's behavior when such axioms are in place. Our characterization includes the linear reinforcement learning rule from Roth and Erev (1995).