Seminars

View all Seminars  |  Download ICal for this event

Towards Next-Generation ML/AI: Robustness, Optimization, Privacy.

Series: Department Seminar

Speaker: Dr. Krishna Pillutla, Google Research, USA

Date/Time: Jan 12 11:00:00

Location: CSA Seminar Hall (Room No. 254, First Floor)

Abstract:
Two trends have taken hold in machine learning and artificial intelligence: a move to massive, general purpose, pre-trained models as well as a move to small, on-device models trained on distributed data. Both these disparate settings face some common challenges: a need for (a) robustness to deployment conditions that differ from training, (b) faster optimization, and (c) protection of data privacy.
As a result of the former trend, large language models have displayed emergent capabilities they have not been trained for. Recent models such as GPT-3 have attained the ability to generate remarkably human-like long-form text. I will describe Mauve, a measure to quantify the goodness of this emergent capability. It measures the gap between the distribution of generated text and that of human-written text. Experimentally, Mauve correlates the strongest with human evaluations of the generated text and can quantify a number of its qualitative properties.

The move to massively distributed on-device federated learning of models opens up new challenges due to the natural diversity of the underlying user data and the need to protect its privacy. I will discuss how to reframe the learning problem to make the model robust to natural distribution shifts arising from deployment on diverse users who do not conform to the population trends. I will describe a distributed optimization algorithm and show how to implement it with end-to-end differential privacy.

To conclude, I will discuss my ongoing efforts and future plans to work toward the next generation of ML/AI techniques by combining the best of both worlds. I will discuss applications ranging from differentially private language models and text generation to decentralized learning.

Speaker Bio:
Krishna Pillutla is a visiting researcher (postdoc) at Google Research, USA in the Federated Learning team. He obtained his Ph.D. at the University of Washington where he was advised by Zaid Harchaoui and Sham Kakade. Before that, he received his M.S. from Carnegie Mellon University and B.Tech. from IIT Bombay where he was advised by Nina Balcan and J. Saketha Nath respectively. Krishna's research has been recognized by a NeurIPS outstanding paper award (2021) and a JP Morgan Ph.D. fellowship (2019-20).

Host Faculty: R Govindarajan