Seminars

View all Seminars  |  Download ICal for this event

Reinforcement Learning Via Sequence Modeling

Series: Department Seminar

Speaker: Dr. Aditya Grover, Assistant Professor of Computer Science University of California, Los Angeles (UCLA)

Date/Time: Mar 04 16:00:00

Location: Microsoft Teams - ON-LINE

Abstract:
I will introduce a framework that abstracts Reinforcement Learning (RL) as a sequence modeling problem. This allows us to draw upon the simplicity and scalability of the Transformer architecture, and associated advances in language modeling such as GPT-x and BERT. I will present Decision Transformer (DT), an architecture that casts the problem of RL as conditional sequence modeling. Unlike prior approaches to RL that fit value functions or compute policy gradients, DT simply outputs the optimal actions by leveraging a causally masked Transformer. By conditioning an autoregressive model on the desired return (reward), past states, and actions, DT can generate future actions that achieve the desired return. I will also present our recent work proposing entropy regularizers to extend DT to online learning with hindsight learning and entropy-based regularization. Despite its simplicity, Decision Transformer matches or exceeds the performance of state-of-the-art model-free offline and online RL baselines on benchmark environments.

Speaker Bio:
Aditya Grover is an assistant professor of computer science at UCLA . His goal is to develop efficient machine learning approaches for probabilistic reasoning under limited supervision, with a focus on deep generative modeling and sequential decision making under uncertainty. He is also an affiliate faculty at the UCLA Institute of the Environment and Sustainability, where he grounds his research in real-world applications in climate science and sustainable energy. His research works have been cited 8000+ times, deployed into production at major technology companies (Instagram, Twitter), and covered in major press venues, such as the Wall Street Journal and Wired. Adityas research has been recognized with two best paper awards (NeurIPS, StarAI), several research fellowships (Google-Simons Institute, Microsoft Research, Lieberman, Adobe), and the ACM SIGKDD doctoral dissertation award. Aditya received his postdoctoral training at UC Berkeley, PhD from Stanford, and bachelors from IIT Delhi, all in computer science.
Microsoft Teams link:
https://tinyurl.com/5n7yfca3

Host Faculty: Dr. Gugan Thoppe