View all Seminars  |  Download ICal for this event

Experiences in Using Reinforcement Learning for Directed Fuzz Testing

Series: M.Tech (Research) Thesis Defense

Speaker: Mr. Subhendu Malakar M.Tech (Research) Student Dept. of CSA

Date/Time: Jan 13 14:00:00

Location: CSA Seminar Hall (Room No. 254, First Floor)

Faculty Advisor: Prof. Vinod Ganapathy

Directed testing is a technique to analyze user-specified target locations in the program. It reduces the time and effort of developers by excluding irrelevant parts of the program from testing and focusing on reaching the target location. Existing tools for directed testing employ either symbolic execution with heavy-weight program analysis or fuzz testing mixed with fine-tuned heuristics. In this thesis, we explore the feasibility of using a data-driven approach for directed testing. We aim to leverage the data generated by fuzz testing tools. We train an agent on the data collected from the fuzzers to learn the optimal mutation for each program input. The agent then directs the fuzzer towards the target location by instructing the optimal action for each program input. We use reinforcement learning based algorithms to train the agent. We implemented a prototype of our approach and evaluated it on synthetic as well as real-world programs. We also evaluate and compare different reward mechanisms to train the agent. Our evaluation shows that an agent based on reinforcement learning can learn the task for simple programs. However, it is not able to perform better for real-world programs as compared to fuzzers that have no such learning agent. From our experiments, we conclude that data-driven approaches are feasible and should be pursued. Although in the present state, reinforcement learning is not able to compete with state of the art fuzzers, we hope that advancements in reinforcement learning will be able to bridge the gap.

Speaker Bio:

Host Faculty: