BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//project/author//NONSGML v1.0//EN
CALSCALE:GREGORIAN
BEGIN:VEVENT
DTEND:20210121T120000Z
UID:f61f444696b6fd1113e03fd30154a7fc-116
DTSTAMP:19700101T120015Z
DESCRIPTION:Dismantling the deep neural network black box
URL;VALUE=URI:https://www.csa.iisc.ac.in/newweb/event/116/dismantling-the-deep-neural-network-black-box/
SUMMARY:Deep neural networks (DNNs) have been quite successful in a variety of supervised learning tasks. A key reason attributed to the success of DNNs is their ability to automatically learn high level representation of the data. The standard view is that low level representations are learnt in the initial layers, and as one proceeds in depth, sophisticated high level representations are learnt in the deeper layers. In this talk, we will focus on DNNs with rectified linear unit (ReLU) activations (ReLU-DNNs), a widely used sub-class of DNNs. We will exploit the gating property of ReLU activations to build an alternative theory for representation learning in ReLU-DNNs. The highlights are:
<br>
1) We encode gating information in a novel neural path feature. We analytically show that the standalone role of gates is characterised by the associated neural path kernel (NPK).
<br>
2) We show via experiments (on standard datasets) that almost all useful information is stored in the gates, and that neural path features are learnt during training.
<br>
3) We show that the neural path kernel has a composite structure. In case of fully connected DNNs, the NPK is a product of the base kernel, in the case of residual networks with skip connections, the NPK has sum of product (of base kernels) form, and in the case of convolutional nets, the NPK is rotationally invariant.
DTSTART:20210121T120000Z
END:VEVENT
END:VCALENDAR