Seminars

View all Seminars  |  Download ICal for this event

HOBIT: Hardness Optimized Batch Sampling for InfoNCE Training

Series: Department Seminar

Speaker: Lokesh Nagalapatti, Microsoft Research

Date/Time: Apr 13 11:30:00

Location: CSA Auditorium, (Room No. 104, Ground Floor)

Abstract:
Contrastive training with InfoNCE loss and in-batch negatives is the standard approach for learning dual-encoder models. Its effectiveness, however, critically depends on the availability of hard negatives; in their absence, learning quickly saturates. Existing methods address this via explicit hard-negative mining, which is often costly or heuristic-driven. We introduce HOBIT, a principled mini-batch construction method that improves in-batch negative quality by reordering training examples at every epoch. HOBIT solves an optimization problem motivated by the InfoNCE objective to yield mini-batches such that each query in the batch is exposed to hard yet non-contradictory, informative negative examples. We show that the optimization objective is monotone and submodular, which in turn leads to a greedy algorithm with the standard (O(1 - 1/e)) approximation guarantee. Empirically, HOBIT incurs negligible computational overhead while significantly outperforming state-of-the-art batching methods, and remains complementary to existing hard-negative mining techniques.

Speaker Bio:
Lokesh is a Senior Researcher at Microsoft Research India, where he is part of the ReFoRM team led by Dr. Manik Varma. His research focuses on information retrieval, with a particular emphasis on developing efficient, scalable, and accurate methods for training dual-encoder models. Prior to joining Microsoft Research, Lokesh was a PhD student at IIT Bombay, advised by Prof. Sunita Sarawagi, where he worked on causal inference and algorithmic recourse. He completed his masters from IISc, working with Prof. M. Narasimha Murty. He has industry experience at IBM Research Labs, Adobe Research, and Samsung.

Host Faculty: Arkaprava Basu