Seminars

View all Seminars  |  Download ICal for this event

Towards Statistical Foundations of Reliable and Defendable Large Language Models

Series: Department Seminar

Speaker: Dr. Subhabrata Majumdar, Co-founder and head of AI, Vijil, USA

Date/Time: Aug 04 10:00:00

Location: CSA Auditorium, (Room No. 104, Ground Floor)

Abstract:
The emergence of Large Language Models (LLMs) has brought in concomitant concerns about the security and reliability of generative AI systems. While LLMs promise powerful capabilities in diverse real-world applications, ensuring that their outputs are resilient to malicious attacks and consistent across similar inputs has significant technical and computational challenges. This situation calls for the revisiting of modern deep learning architectures through a statistical lens.

I will present on two interconnected themes in this area. First, I will introduce Representation Noising (RepNoise), a defense mechanism that protects the weights of open-source LLMs against malicious uses. RepNoise achieves this through controlled noise injection in the knowledge representations inside a model that makes it harder to recover harmful information later. Second, I will discuss my work on the consistency problem - the equivalent of robustness in LLMs - concerned with measuring and minimizing the sensitivity of LLM outputs to input variations through a combination of controlled synthetic data generation and fine-tuning.

I will conclude by discussing ongoing work in both areas, including the development of theoretical bounds for the strength of defense mechanisms like RepNoise, and robustness frameworks for ensuring reliability of AI agents in high-stakes applications.

Speaker Bio:
Subho Majumdar is co-founder and head of AI at Vijil, a US-based startup that helps enterprises build and operate trustworthy AI agents. Previously, he was a senior scientist in the security research team at Splunk and the Data Science and AI Research team at AT&T Labs. He has pioneered the use of trustworthy AI methods in multiple companies, wrote a book, and founded multiple nonprofit efforts in this area. He is a recipient of the International Indian Statistical Association (IISA) Early Career Award in Statistics and Data Sciences. His research interests are on the security and reliability of LLMs and statistical machine learning.

Host Faculty: R Govindarajan