View all Seminars  |  Download ICal for this event

Novel Reinforcement Learning Algorithms and Applications to Hybrid Control Design Problems.

Series: M.Tech (Research)Thesis Defence - ONLINE

Speaker: Mr. Meet Pradhuman Gandhi, M.Tech (Research) student, Dept. of CSA

Date/Time: Oct 26 11:00:00

Location: Microsoft Teams - ONLINE

Faculty Advisor: Prof. Shalabh Bhatnagar

The thesis is a compilation of two independent works. In the first work, we develop novel framework that allows assignment of different weights to different $n$ step returns, which helps us develop several schedule based algorithms. Learning the value function of a given policy from the data samples is an important problem in Reinforcement Learning.
TD($lambda$) is a popular class of algorithms to solve this problem. However, the weight assigned to different $n$-step returns decreases exponentially with increasing $n$ in TD($lambda$). Here, we present a $lambda$-schedule procedure that allows flexibility in weight assignment to the different $n$-step returns.
Based on this procedure, we propose an on-policy algorithm, TD($lambda$)-schedule, and an off-policy algorithm, TDC($lambda$)-schedule, respectively. We provide proofs of almost sure convergence for both algorithms under a general Markov noise framework as well as present the results of experiments where these algorithms are seen to show improved performance.
In the second work, we design hybrid control policies for hybrid systems whose mathematical models are unknown.

Our contributions are threefold here.
First, we propose a framework for modelling the hybrid control design problem as a single Markov Decision Process (MDP).
This result facilitates the application of off-the-shelf algorithms from Reinforcement Learning (RL) literature towards designing optimal control policies.
Second, we model a set of benchmark examples of hybrid control design problem in the proposed MDP framework.
Third, we adapt the recently proposed Proximal Policy Optimisation (PPO) algorithm for the hybrid action space and apply it to the above set of problems. It is observed that in each case the algorithm converges and finds the optimal policy.
Microsoft teams link:

Speaker Bio:

Host Faculty: