Journal of Economic Literature (JEL) Code(s)
C1, C5, C9
What is the most statistically eﬀicient way to do oﬀ-policy optimization with batch data from bandit feedback? For log data generated by contextual bandit algorithms, we consider oﬀline estimators for the expected reward from a counterfactual policy. Our estimators are shown to have lowest variance in a wide class of estimators, achieving variance reduction relative to standard estimators. We then apply our estimators to improve advertisement design by a major advertisement company. Consistent with the theoretical result, our estimators allow us to improve on the existing bandit algorithm with more statistical conﬁdence compared to a state-of-theart benchmark.
Narita, Yusuke; Yasui, Shota; and Yata, Kohei, "Efficient Counterfactual Learning from Bandit Feedback" (2018). Cowles Foundation Discussion Papers. 110.