Information fusion for robust detection of scarce features/objects in high resolution electro-optical satellite imagery

Abstract

We conducted research to develop and test methods to improve the detection of scarce objects in high-resolution electro-optical satellite imagery. We demonstrated improvements through various forms of information and data fusion that included heuristic analyses, fuzzy integrals, and neural learning. Scarce objects of interest included Surface-to-Air Missile (SAM) Sites, SAM Launch Pads (SAM LP), SAM Transporter Erectile Launchers (SAM TELs), Construction Sites, and Engineering Vehicles. We demonstrated improved detection and reduced error in broad area search for SAM Sites in Southeast China by fusing the larger feature with the detection of smaller, multi-scale elements/components (i.e. SAM TELs and LPs) within the larger feature using heuristics and neural learning. We expanded these techniques to demonstrate improved detection of Construction Sites. We next demonstrated the improved detection of small, scarce objects (i.e. Engineering Vehicles) by fusing the outputs from multiple deep CNNs of various architectures and sizes through multi-layer perceptrons and fuzzy integrals. Major improvements were then achieved by leveraging existing CNN models designed for 3-band (RGB) models to train and process 8-band multi-spectral imagery partitioned into a set of three 3-band images and then fusing the results using fuzzy integrals. We finally demonstrated that bounding-box object detection for SAM TELs could be improved by ensembling/fusing bounding-box results from multiple detectors using a novel Pseudo-Cell State Long-Short Term Memory (PCS-LSTM) neural network developed in this research. The PCS-LSTM is a two-layer, non-serial LSTM neural network architecture that utilizes bounding-box context to act as a first-layer quasi/pseudo cell memory.