A study of remote sensing based natural and built environment monitoring: from fully supervised to weakly supervised learning

Abstract

Remote Sensing (RS) has been one of the pivotal technologies in the monitoring of natural and built environments and has been widely applied for disaster monitoring, damage detection for civil structures and infrastructure, impact assessment, and many decision-making tasks. In this dissertation, I focus on floods, which, as a type of natural hazard, afflict both natural and built environments on a global scale. As I navigate through the literature, two technical contributions have been achieved in my work. I first synthesize the latest advances in optical Remote Sensing, summarizing its advantages and usage by employing the classical supervised learning methodology. Through this practice, the operational readiness of adopting optical RS technologies for flood disaster management is analyzed, and technical challenges are identified. In addition, I fill a technical gap in past RS and ML practices for understanding disasters, which have ignored the dual facets of the disasters—their hazardous characteristics and their disastrous effects. To address this, I systematically evaluate which hazard types and damage levels are more easily detectable in RS images. Among the many technical challenges in RS practice is the extreme scarcity of granular labeling relative to the volume of imagery data. More specifically, there are very few RS datasets that provide pixel-level annotations. If one aims to perform image-based segmentation using a fully supervised approach, two methods have been adopted, as demonstrated in my previous research endeavors described above: (1) the use of a small labeled dataset, often prepared individually; and (2) targeting ad-hoc problems at regional scales, hence using images from the regions of interest to minimize the complexity and variety of observed scenes in images. On the other hand, the RS community is distinct from the rest of the computer vision and image understanding communities in one particular aspect — the rich presence of regional-level human-produced semantic information, including labels, names, and even captions, which is the outcome of years of using RS data to generate mapping products. To address this challenge and to exploit this advantage, I propose a Weakly Supervised Learning (WSL) framework for realizing Semantic Segmentation and RS-based mapping. In this dissertation, WSL is theoretically explored in the context of modern deep learning. A unique transformer-based architecture is developed, further integrated with a novel set of contrastive loss functions, to realize WSL-based segmentation using an imagery dataset with image-only labels. When applied to the task of monitoring global floods, it offers the advantage of reducing the labor-intensive process of labeling RS data at the pixel level. Additionally, this approach promises to provide a more rapid assessment of their impact by integrating with real-time flood alert systems. To further expand its application arena, I explore the feasibility of transfer learning for the purpose of global coastal ship detection. In summary, this dissertation proposal seeks to advance the field of deep learning technologies and their applications for remote sensing-based monitoring of global natural and built environments. More specifically, in the area of flood disaster response, the proposed technical notions of operational readiness, understanding of disaster effects in RS images, and the WSL methodology for RS-based mapping have partially bridged the gap between flood forecasting and understanding their consequences, thereby facilitating the progress of achieving flood disaster-resilient communities at a global scale.