Seminars

View all Seminars  |  Download ICal for this event

HyCache: Hybrid Caching for Accelerating DNN Training Input Pipelines

Series: M.Tech (Research) Thesis Defense

Speaker: Keshav Vinayak Jha

Date/Time: Aug 07 10:00:00

Location: CSA Lecture Hall (Room No. 112, Ground Floor)

Faculty Advisor: Arkaprava Basu

Abstract:
The performance of deep neural network (DNN) training is a function of both the training compute latency and the latency to fetch and preprocess the input data needed to train the model. With advancements in GPUs and ML software platforms, the training performance on GPUs has seen substantial performance gains. These improvements have shifted the bottleneck to the CPU-based input pipeline, which preprocesses and transforms data before it is fed into the GPUaccelerator for training.

To accelerate the input pipeline, some prior works cache intermediate data generated from an input processing step and reuse the cached data rather than recompute that step. Prior works cache the data either in memory or in storage and require the entire output generated from a step to be cached. Given that modern training systems are equipped with substantial memory and storage, exploiting and optimizing both capacities is crucial. In this paper, we propose Hybrid Cache(HyCache), a technique that enables the caching of a subset of tensors from multiple intermediate steps on both memory and storage automatically, without programmer involvement. HyCache has the ability to partially cache the preprocessing step outputs across both memory and storage capacities to maximize resource utilization while allowing recomputation of the uncached data as well. HyCache provides a user-friendly library interface that determines the optimal tradeoff between recomputing a preprocessing step and caching across memory and storage. HyCache outperforms state-of-the-art prior approaches, delivering a raw pipeline throughput improvement ranging from 1.11? to 10.1x the typical preprocessing pipeline.

Speaker Bio:
Keshav Vinayak Jha is an MTech Research student in the Department of Computer Science and Automation at IISc, Bangalore. He is part of the Computer Systems Lab (CSL) and works under the guidance of Prof. Arkaprava Basu. His primary area of research is finding and alleviating potential system bottlenecks to speed up AI training applications.

Host Faculty: Arkaprava Basu