Seminars

View all Seminars  |  Download ICal for this event

Uncertainty Quantification for Large Language Models

Series: Bangalore Theory Seminars

Speaker: Shubhendu Trivedi, MIT

Date/Time: Nov 07 17:00:00

Location: Online Talk (See Teams link below)

Abstract:
The rise of large language models (LLMs) has significantly advanced the state-of-the-art across a range of natural language generation tasks. However, deploying LLM-based compound systems for specific applications remains a challenging endeavor. For such systems to be applied reliably, we need accurate measures of uncertainty and confidence of LLM outputs. Here, uncertainty denotes the -dispersion- of possible predictions for a given input, while confidence pertains to the reliability of a particular input-generation pair. This talk will first define these basic notions and introduce simple baselines for confidence scoring and uncertainty quantification, and then discuss recent developments in the field, including fine-tuning techniques and conformal prediction methods.


Microsoft teams link:

Link

We are grateful to the Kirani family (Link and the Walmart Center for Tech Excellence (Link for generously supporting this seminar series


Hosts: Rameesh Paul, KVN Sreenivas, Rahul Madhavan, Debajyoti Kar