Seminars

View all Seminars  |  Download ICal for this event

Reinforcement Learning via Stochastic Approximation

Series: Distinguished Seminar

Speaker: Prof. M. Vidyasagar, FRS SERB-National Science Chair and Distinguished Professor IIT Hyderabad

Date/Time: Oct 15 14:30:00

Location: CSA Seminar Hall (Room No. 254, First Floor)

Abstract:
Reinforcement Learning (RL) is one of the most popular branches of AI/ML. Moreover, many though not all of the currently popular RL algorithms have a solid mathematical justification. At present, a wide variety of proof methods are used to study RL algorithms. Therefore, there is scope for some unification of these diverse approaches. In this talk, I will show how a large fraction of current RL algorithms can be viewed as implementations of the Stochastic Approximation (SA) algorithm, in various forms. I will also show that, by using martingale methods, the convergence of these SA variants can be established more simply and with fewer assumptions than is possible using the ODE approach, which is another popular method of analyzing SA algorithms. Some problems for future research will also be indicated.

Speaker Bio:
Prof. Mathukumalli Vidyasagar was born in Guntur, India on September 29, 1947. He received the B.S., M.S. and Ph.D. degrees in electrical engineering from the University of Wisconsin in Madison, in 1965, 1967 and 1969 respectively. Between 1969 and 1989, he was a Professor of Electrical Engineering at Marquette University, Concordia University, and the University of Waterloo. In 1989 he returned to India as the Director of the newly created Centre for Artiβicial Intelligence and Robotics (CAIR) in Bangalore. under the Ministry of Defence, Government of India. Between 1989 and 2000, he built up CAIR into a leading research laboratory with about 40 scientists and a total of about 85 persons, working in areas such as βlight control, robotics, neural networks, and image processing. In 2000 he moved to the Indian private sector as an Executive Vice President of Indias largest software company, Tata Consultancy Services. In 2009 he retired from TCS and joined the Erik Jonsson School of Engineering & Computer Science at the University of Texas at Dallas, as a Cecil & Ida Green Chair in Systems Biology Science. In January 2015 he received the Jawaharlal Nehru Science Fellowship and divided his time between UT Dallas and the Indian Institute of Technology Hyderabad. In January 2018 he stopped teaching at UT Dallas and now resides full-time in Hyderabad. He continues his association with IIT Hyderabad. Since March 2020, he is a SERB National Science Chair. His current research is in the area of Reinforcement Learning, with emphasis on using stochastic approximation theory. More broadly, he is interested in machine learning, systems and control theory, and their applications. Until recently he was exploring the area of compressed sensing. On the applications front, he is interested in applying ideas from machine learning to problems in computational biology with emphasis on cancer. Vidyasagar has received a number of awards in recognition of his research contributions, including Fellowship in The Royal Society, the worlds oldest scientiβic academy in continuous existence, the IEEE Control Systems (Technical Field) Award, the Rufus Oldenburger Medal of ASME, the John R. Ragazzini Education Award from AACC, and others. He is the author of thirteen books and more than 150 papers in peer-reviewed journals.

Host Faculty: Prof. Shalabh Bhatnagar