Seminars

View all Seminars  |  Download ICal for this event

PyGraph: Compiler Support for Efficient and Transparent Use of CUDA Graphs

Series: M.Tech (Research) Colloquium

Speaker: Abhishek Ghosh

Date/Time: Jan 17 14:30:00

Location: CSA Lecture Hall (Room No. 112, Ground Floor)

Faculty Advisor: Arkaprava Basu

Abstract:
CUDA Graphs --- a recent hardware feature introduced for NDIVIA GPUs --- aim to reduce CPU launch overhead by capturing and launching a series of GPU tasks (kernels) as a DAG.
However, deploying CUDAGraphs faces several challenges today due to the static structure of a graph. It also incurs performance overhead due to data copy. In fact, we show a counter-intuitive result --- deploying CUDA Graphs hurts performance in many cases.

We introduce PyGraph, a novel approach to automatically harness the power of CUDA Graphs within PyTorch2.
Driven by three key observations, PyGraph embodies three novel optimizations: it enables wider deployment of CUDA Graphs, reduces GPU kernel parameter copy overheads, and selectively deploys CUDA Graphs based on a cost-benefit analysis.
PyGraph seamlessly integrates with PyTorch2\'s compilation toolchain, enabling efficient use of CUDA Graphs without manual modifications to the code. We evaluate PyGraph across various machine learning benchmarks, demonstrating substantial performance improvements over PyTorch2.

Speaker Bio:
Abhishek Ghosh is an MTech Research student in the Department of Computer Science and Automation at IISc, Bangalore. He is a member of the Computer Systems Lab (CSL) and works under the guidance of Prof. Arkaprava Basu. His primary area of research focuses on advancing compiler technologies for machine learning workloads by optimizing the GPU software stack at the hardware-software interface to enhance performance and efficiency.

Host Faculty: Arkaprava Basu