Seminars

View all Seminars  |  Download ICal for this event

Safe Policy Learning from Offline Data: Revisiting Policy Extraction and Constraint Satisfaction

Series: Department Seminar

Speaker: Dr. Raunak Bhattacharyya, Yardi School of Artificial Intelligence, Indian Institute of Technology Delhi

Date/Time: Oct 27 11:00:00

Location: CSA Auditorium, (Room No. 104, Ground Floor)

Abstract:
Reinforcement learning offers a powerful framework for sequential decision-making, but learning effective policies while satisfying safety requirements remains a fundamental challenge. Safe reinforcement learning incorporates safety requirements into the learning process, often through constrained Markov decision processes (CMDPs). When collecting new data through interaction is expensive or potentially unsafe, offline reinforcement learning offers an alternative by learning from previously collected datasets. Learning safe policies from offline data introduces challenges in both constraint satisfaction and policy extraction. Existing approaches often rely on soft constraint satisfaction, which may not guarantee that safety requirements are met. At the same time, extracting effective policies from offline data remains challenging, and the computational cost of policy inference can limit the efficiency of learned policie s. In this talk, we will revisit how policies are extracted and how safety constraints are satisfied in safe offline reinforcement learning. We will show that reachability-based identification of feasible behaviors, combined with behavior-constrained policy extraction, offers a promising direction towards improving both safety and policy performance. We will also examine how these approaches challenge existing assumptions about the trade-off between satisfying safety constraints and learning effective policies. This talk is based on joint work with Mahesh Keswani, Parth Bhardwaj, and Ankur Kumar.

Speaker Bio:
Raunak P. Bhattacharyya is an Assistant Professor in the Yardi School of Artificial Intelligence at IIT Delhi. His research focuses on learning safe and effective behaviors for decision making in uncertain environments. Before joining IIT Delhi in 2024, he was a postdoctoral researcher at the University of Oxford. He earned his Ph.D. from Stanford University in 2021. He obtained his M.S. from Georgia Tech, and his B.Tech. from IIT Bombay. 

Host Faculty: Prof. Shalabh Bhatnagar