# Contextual Bandits under Delayed Feedback

**Authors:** Claire Vernade, Alexandra Carpentier, Giovanni Zappella, Beyza Ermis, Michael Brueckner  
**arXiv:** [1807.02089](https://arxiv.org/abs/1807.02089)  
**Date:** 2018-07-05  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/1807.02089v1/

## Key points

- The paper addresses the issue of delayed feedback in industrial systems using bandit algorithms
- Delayed feedback makes it challenging for the learner to distinguish between negative and positive signals that have not yet been received
- The authors propose a new UCB-based algorithm for contextual bandits with delayed and censored feedback
- The algorithm takes into account potential delay and censorship of feedback
- Finite time regret analysis is provided to quantify the performance of their algorithm compared to an optimal strategy
- Empirical evaluation shows the superiority of their algorithm in terms of performance and efficiency compared to existing approaches
- This research contributes to improving decision-making processes in various industrial systems by addressing the issue of delayed feedback in contextual bandit problems.

## Summary

The paper titled "Contextual Bandits under Delayed Feedback" addresses the issue of delayed feedback in industrial systems that employ bandit algorithms. These systems aim to optimize binary indicators, such as clicks. However, when the reward is not sent immediately, it becomes challenging for the learner to distinguish between a negative signal and a positive one that has not yet been received. This ambiguity can lead to the learner waiting indefinitely for feedback that may never arrive. To tackle this problem, the authors propose a new UCB-based algorithm for contextual bandits with delayed and censored feedback. They define and address this specific variant of the contextual bandit problem by developing an algorithm that takes into account the potential delay and censorship of feedback. To demonstrate the effectiveness of their proposed algorithm, the authors provide a finite time regret analysis which quantifies how much worse their algorithm performs compared to an optimal strategy. Additionally, they conduct an empirical evaluation comparing their algorithm against a commonly used baseline in practice. The findings of their empirical evaluation highlight the superiority of their UCB-based algorithm in terms of performance and efficiency compared to existing approaches. By addressing the issue of delayed feedback in contextual bandit problems, this research contributes to improving decision-making processes in various industrial systems.