Fast and robust deep neural networks design
No Thumbnail Available
Authors
Meeting name
Sponsors
Date
Journal Title
Format
Thesis
Subject
Abstract
In the past few years, we have witnessed a rapid development of deep neural networks in computer vision, from basic image classification tasks to some more advanced applications e.g. object detection and semantic segmentation. Inspire of its great success, there exists two challenges of deep neural networks real-world applications: its computational cost and vulnerability. Thus we are aimed to deal with these two problems in this thesis. To speed up deep networks, we propose a L₁-Norm based low-rank approximation method to reduce oat operations based on the alternating direction method (ADM) in Chapter 2. Our experimental results on public datasets, including CIFAR-10 and ImageNet, demonstrate that this new decomposition scheme outperforms the recently developed L₂-norm based nonlinear decomposition method. To defend against adversarial examples, we develop a novel pre-processing algorithm based on image restoration to remove adversarial attack noise in Chapter 3. We detect high-sensitivity which have significant contributions to the image classification performance. Then we partition the image pixels into the two groups: high-sensitivity and low-sensitivity keypoints. For the low-sensitivity pixels, we use the existing total variation (TV) norm-based image smoothing. For the high-sensitivity pixels, we develop a structure-preserving low-rank image completion methods. Based on matrix analysis and optimization, we have derived an iterative solution for this optimization problem. This high-sensitivity points detection helps us to improve the defense against white-box attack BPDA. However, in our keypoints defense we only remove and recover a few part of pixels, which indicates there are still many perturbation over the whole image. In Chapter 4, we propose a novel image completion algorithm structure-preserving progressive lowrank image completion (SPLIC ) based on smoothed rank function (SRF) in which we can reconstruct a image with over 50% removed pixels. In SPLIC, we randomly remove over 50% pixels on the image and then do matrix completion by low-rank approximation to remain the global structure of the image. Differ from other lowrank methods, we replace nuclear norm by smoothed rank function (SRF) for its closer rank function approximation. We introduce total variance (TV) regularization to improve image reconstruction, and then combine total variance (TV) norm de-noising to further remove the perturbation over the whole image. Then we train the network on the SPLIC images. The experimental results show our SPLIC outperforms other pre-processing methods in image reconstruction, gray-box and black-box scenario.
Table of Contents
DOI
PubMed ID
Degree
Ph. D.
Thesis Department
Rights
OpenAccess.
License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 License. Copyright held by author.