Vision task driven image super-resolution and image enhancement
No Thumbnail Available
Authors
Meeting name
Sponsors
Date
Journal Title
Format
Subject
Abstract
In visual object recognition problems, low-light exposure, and low-quality images present great challenges in a variety of navigation and surveillance use cases. Recent advancements in deep learning-based methods may contribute towards the enhancement of low-light images to high-quality images with enough exposure. However, these pixel domain signal recovery metrics may not directly correlate to the machine vision tasks like key points detection and object recognition, resulting in loss of performance. We develop a Scale-Invariant Feature Transform (SIFT) detection task-driven dark image enhancement method that learns the difference of Gaussian (DoG) pyramid from dark image input directly, with a cascade network that re-uses the network weights learned at different scales. Simulation results demonstrate that this type of vision task loss-driven learning improves the overall performance vis-a-vis pixel recovery and then learning framework. Low-resolution images also present challenges to a variety of object recognition problems in a variety of surveillance and navigation applications. Inspired by the recent advances of deep convolutional neural networks in general image SR tasks, we develop a computer vision task-driven image SR solution by learning super-resolved gradient images using multiple convolutional neural networks for different scales. Recovering super-resolved gradient images at multiple scales, enables the system to recover more information useful for high level vision tasks than simply SR in the pixel domain. In particular, we propose a residual learning framework to perform image SR in the Difference of Gaussian (DOG) domain. The trained residual network models are then adapted to drive a widely adopted key point algorithm for image recognition, i.e. the SIFT detection and matching. Experimental results show that the proposed approach can significantly improve the SIFT keypoints repeatability compared to the state of the art in pixel domain image SR solutions. However, using multi-frame, super-resolution algorithm can reconstruct high- resolution images by incorporating the information of the subsequent images. Most of the super-resolution techniques for multi-frames either use a more traditional or mathematical approach or deep learning-based approach with optical flow in consideration. We develop a way to combine the optical flow enabled sub-pixel registration method for mapping into the high-resolution grid and a deep residual learning approach for restoring features with noise removal. The results exhibit a significant gain over the state of art methods and the bi-cubic interpolation method.
Table of Contents
Introduction -- Background study and literature review -- Gradient image super-resolution -- Dark image enhancement -- Multi-frame super-resolution -- Conclusion and future work -- Appendix
DOI
PubMed ID
Degree
Ph.D. (Doctor of Philosophy)