Seminars

View all Seminars  |  Download ICal for this event

Online Learning with Markovian Data via Reverse Experience Replay

Series: Department Seminar

Speaker: Dr. Prateek Jain, Research Scientist at Google Research India, &Adjunct Faculty member at IIT Kanpur

Date/Time: Feb 28 16:00:00

Location: Microsoft Teams - ON-LINE

Abstract:
Learning with Markovian data is a challenging problem with several applications in critical domains like reinforcement learning, control theory, time series analysis etc. Techniques like SGD are the workhorse of large-scale learning, with rigorous analysis for streaming i.i.d. data in several regimes. But their applicability to non i.i.d. Markovian data is unclear due to dependency between points.<br>
In this talk, we will present results showing that SGD in general can be significantly sub-optimal for Markovian data. In contrast, through three critical problems: a) linear regression, b) dynamical system identification aka vector auto-regressive model estimation, c) policy learning with Linear MDPs, we demonstrate that SGD enhanced with experience replay--a popular heuristic used in RL literature--leads to nearly optimal solutions. To the best of our knowledge, we provide the first rigorous analysis of the practically popular experience replay technique. Similarly, our result provides the first provably efficient Q-learning style method for finding optimal policy for linear MDPs.<br>
Based on joint works with Naman Agarwal, Syomantak Chaudhuri, Suhas Kowshik, Dheeraj Nagaraj, Praneeth Netrapalli, Carrie Wu.
<br>
<br>
After the talk we will then have a 30 mins Q&A session with students to make them aware of various opportunities at Google Research India including predoc roles and student researchership.<br>
-------------------------------------------------------------------------<br>
Google research India has several opportunities for graduating/final year students to participate in research projects. Here is a partial list:<br>
1. Predoc researcher: Candidates who hold Bachelors/Masters degrees can spend up to two years working with research teams at Google research India on cutting-edge research projects. Most of our past pre-doc researchers have then gone on to pursue PhDs at top schools such as MIT, Berkeley, CMU, University of Washington etc.<br>
2. Student researcher/ student internships: This is for students who are in their final/pre-final year of their Bachelors/Masters or any year of their PhD program to spend part of their time working on research projects with teams at Google research India.

Speaker Bio:
Prateek Jain is a research scientist at Google Research India and an adjunct faculty member at IIT Kanpur. Earlier, he was a Senior Principal Researcher at Microsoft Research India. He obtained his PhD degree from the Computer Science department at UT Austin and his BTech degree from IIT Kanpur. He works in the areas of large-scale and non-convex optimization, high-dimensional statistics, and ML for resource-constrained devices. He wrote a monograph on Non-convex Optimization in Machine Learning summarizing many of his results in non-convex optimization. Prateek regularly serves on the senior program committee of top ML conferences and is an action editor for JMLR, and an associate editor for SIMODS. He has also won ICML-2007, CVPR-2008 best student paper award and more recently his work on alternating minimization has been selected as the 2020 Best Paper by the IEEE Signal Processing Society. Prateek also received the Young Alumnus Award 2021 from IIT Kanpur and received the ACM India Early Career Researcher Award for 2021.
Microsoft teams link:
https://tinyurl.com/58ez7deh

Host Faculty: Prof. Chiranjib Bhattacharyya