Seminars

View all Seminars  |  Download ICal for this event

Experiences in Using Reinforcement Learning for Directed Fuzz Testing

Series: M.Tech (Research) Thesis Defense

Speaker: Mr. Subhendu Malakar M.Tech (Research) Student Dept. of CSA

Date/Time: Jan 13 14:00:00

Location: CSA Seminar Hall (Room No. 254, First Floor)

Faculty Advisor: Prof. Vinod Ganapathy

Abstract:
Directed testing is a technique to analyze user-specified target locations in the program.
It reduces the time and effort of developers by excluding irrelevant parts of the program
from testing and focusing on reaching the target location. Existing tools for directed testing
employ either symbolic execution with heavy-weight program analysis or fuzz testing mixed
with fine-tuned heuristics.

In this thesis, we explore the feasibility of using a data-driven approach for directed testing.
We aim to leverage the data generated by fuzz testing tools. We train an agent on the
data collected from the fuzzers to learn the optimal mutation for each program input. The
agent then directs the fuzzer towards the target location by instructing the optimal action for
each program input. We use reinforcement learning based algorithms to train the agent. We
implemented a prototype of our approach and evaluated it on synthetic as well as real-world
programs. We also evaluate and compare different reward mechanisms to train the agent.
Our evaluation shows that an agent based on reinforcement learning can learn the task
for simple programs. However, it is not able to perform better for real-world programs as
compared to fuzzers that have no such learning agent. From our experiments, we conclude
that data-driven approaches are feasible and should be pursued. Although in the present
state, reinforcement learning is not able to compete with state of the art fuzzers, we hope
that advancements in reinforcement learning will be able to bridge the gap.