Seminars
View all Seminars | Download ICal for this eventSemi-Bandit Learning for Monotone Stochastic Optimization
Series: Bangalore Theory Seminars
Speaker: Arpit Agarwal Indian Institute of Technology Bombay
Date/Time: Dec 24 11:30:00
Location: CSA Auditorium, (Room No. 104, Ground Floor)
Abstract:
Stochastic optimization is a widely used approach for optimization under uncertainty, where uncertain input parameters are modeled by random variables. Exact or approximation algorithms have been obtained for several fundamental problems in this area. However, a significant limitation of this approach is that it requires full knowledge of the underlying probability distributions. Can we still get good (approximation) algorithms if these distributions are unknown, and the algorithm needs to learn them through repeated interactions? Particularly, can we design an online learning algorithm whose performance smoothly approaches the performance of an (offline) optimal algorithm which has knowledge of these distributions?
In this work, we resolve this question for a large class of ??monotone? stochastic problems, by providing a generic online learning algorithm with sqrt{T log T} regret relative to the best approximation algorithm (under known distributions). Importantly, our online algorithm works in a semi-bandit setting, where in each period, the algorithm only observes samples from the r.v.s that were actually probed. Our framework applies to several fundamental problems in stochastic optimization such as prophet inequality, Pandoras box, stochastic knapsack, stochastic matchings and stochastic submodular optimization.
In a series of lectures, I will first introduce the multi-armed bandits framework and describe the popular algorithmic principle of