BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//project/author//NONSGML v1.0//EN
CALSCALE:GREGORIAN
BEGIN:VEVENT
DTEND:20230112T120000Z
UID:b154bbbb1e7f2f156c6b916814993be6-385
DTSTAMP:19700101T120011Z
DESCRIPTION:Towards Next-Generation ML/AI: Robustness, Optimization, Privacy.
URL;VALUE=URI:https://www.csa.iisc.ac.in/newweb/event/385/towards-next-generation-ml-ai-robustness-optimization-privacy/
SUMMARY:Two trends have taken hold in machine learning and artificial intelligence: a move to massive, general purpose, pre-trained models as well as a move to small, on-device models trained on distributed data. Both these disparate settings face some common challenges: a need for (a) robustness to deployment conditions that differ from training, (b) faster optimization, and (c) protection of data privacy.
As a result of the former trend, large language models have displayed emergent capabilities they have not been trained for. Recent models such as GPT-3 have attained the ability to generate remarkably human-like long-form text. I will describe Mauve, a measure to quantify the goodness of this emergent capability. It measures the gap between the distribution of generated text and that of human-written text. Experimentally, Mauve correlates the strongest with human evaluations of the generated text and can quantify a number of its qualitative properties.

The move to massively distributed on-device federated learning of models opens up new challenges due to the natural diversity of the underlying user data and the need to protect its privacy. I will discuss how to reframe the learning problem to make the model robust to natural distribution shifts arising from deployment on diverse users who do not conform to the population trends. I will describe a distributed optimization algorithm and show how to implement it with end-to-end differential privacy.

To conclude, I will discuss my ongoing efforts and future plans to work toward the next generation of ML/AI techniques by combining the best of both worlds. I will discuss applications ranging from differentially private language models and text generation to decentralized learning.
DTSTART:20230112T120000Z
END:VEVENT
END:VCALENDAR