Seminars

View all Seminars  |  Download ICal for this event

Improving the quality of GPU programs

Series: Ph.D. Colloquium

Speaker: Nayak Ajay Ashok

Date/Time: Aug 06 16:00:00

Location: CSA Auditorium, (Room No. 104, Ground Floor)

Faculty Advisor: Arkaprava Basu

Abstract:
The ability of Graphics Processing Units (GPUs) to efficiently execute thousands of threads in parallel has made GPUs a crucial component of modern software systems, accelerating workloads in machine learning and high-performance computing. As GPU-accelerated programs become increasingly common, their quality affects a large swath of today\'s software.

In this thesis, we study the quality of GPU programs with respect to two properties: correctness and performance. Correctness requires that programs avoid subtle software bugs that can lead to unexpected program outcomes, silently corrupt data, or cause programs to crash. Performance requires that programs avoid semantic inefficiencies (i.e., bugs), which can leave performance on the table. These are complementary properties: the performance of an incorrect program has no value. Thus, GPU programs must be correct and performant.

A high-quality GPU program must therefore avoid bugs that affect correctness and performance. However, identifying such bugs is challenging in GPU programs, where they can depend on various factors, such as parallel execution and hardware behavior.

This thesis presents three works to improve the quality of GPU programs. The first two works focus on correctness. Subtle correctness bugs, such as data races and memory safety violations, pose a major threat to the correctness of GPU programs. Unfortunately, existing methods to detect them have several shortcomings, which we address in our work. First, we build HiGRD to improve the accuracy of GPU data race detection, ie{} detecting true races and minimizing false alarms. Existing dynamic (runtime) techniques often miss true races, as they require races to manifest during testing. Static (compile-time) techniques use GPU function (kernel) level reasoning to detect such races. However, they generate numerous false alarms by considering infeasible executions. HiGRD addresses these limitations by extracting constraints, such as GPU kernel parameters and thread grid dimensions, at compile time and combining them with GPU kernel-level reasoning. This allows accurate race detection while avoiding false alarms caused by infeasible executions. Compared to prior compile-time techniques, HiGRD also supports fine-grained synchronization methods such as locks.

Second, we build SCuBA to detect elusive memory safety bugs in GPU programs. Existing techniques for detecting memory safety bugs are dynamic. They report bugs only when they manifest in a program, i.e., the technique is input-dependent. We observe that a memory safety bug can occur if there is no relationship between the variables used to allocate memory and those used to calculate the memory access offset. SCuBA analyzes the relation between variables used to decide the allocation sizes to memory access offsets, enabling compile-time detection of input-dependent bugs. Unlike previous works, it detects out-of-bound (OOB) accesses, which can occur within an allocation of dynamic shared memory. Here, an access remains within an allocation but crosses the logical boundary of an intended data structure.

The third work of this thesis focuses on the performance of GPU programs. Performance is the central motivation for using GPUs. Avoidable inefficiencies (a.k.a, performance bugs) can waste expensive hardware resources and reduce the benefit of GPU acceleration. Widely used GPU profilers such as NVIDIA Nsight, are useful for identifying inefficiencies, but they do not provide actionable advice to eliminate the inefficiency.

We therefore build ScopeAdvice that identifies over-synchronization in GPU programs and provides actionable advice to the programmers. Over-synchronization is a performance bug where a program uses a stronger than necessary or redundant synchronization operation to preserve correctness. Programmers are often conservative in deciding the synchronization operations in GPU programs, causing over-synchronization. This preserves program correctness, but can leave significant performance on the table. ScopeAdvice reasons about different synchronization primitives and GPU memory-system behavior, and provides actionable advice to programmers for eliminating over-synchronization and improving performance.

Speaker Bio:
Ajay is a Ph.D. candidate in the Department of Computer Science and Automation at the Indian Institute of Science (IISc), advised by Dr. Arkaprava Basu. His current research focuses on the correctness and performance of GPU programs. He is the recipient of the prestigious Prime Minister\'s fellowship for Doctoral research 2022-2026. He has co-authored papers published at top-tier venues like MICRO, ASPLOS and OSDI. Previously, he obtained his Master\'s from IISc and Bachelor\'s from NIE, Mysuru.

Host Faculty: Arkaprava Basu