ICE-MILK: Intelligent Crowd Engineering using Machine-based Internet of Things Learning and Knowledge Building

Abstract

The lack of proper crowd safety control and management often leads to spreading human casualties and infectious diseases (e.g., COVID-19). Many Machine Learning (ML) technologies inspired by computer vision and video surveillance systems have been developed for crowd counting and density estimation to prevent potential personal injuries and deaths at densely crowded political, entertaining, and religious events. However, existing crowd safety management systems have significant challenges and limitations on their accuracy, scalability, and capacity to identify crowd characterization among people in crowds in real-time, such as a group characterization, impact of occlusions, mobility and contact tracing, and distancing.

In this dissertation, we propose an Intelligent Crowd Engineering platform using Machine-based Internet of Things Learning, and Knowledge Building approaches (ICE-MILK) to enhance the accuracy, scalability, and crowd safety management capacity in real-time. Specifically, we design an ICE-MILK structure with three critical layers: IoT-based mobility characterization, ML-based video surveillance, and semantic information-based application layers. We built an IoT-based mobility characterization system by predicting and preventing potential disasters through real-time Radio Frequency (RF) data characterization and analytics. We tackle object group identification, speed, direction detection, and density for the mobile group among the many crowd mobility characteristics. Also, we tackled an ML-based video surveillance approach for effective dense crowd counting by characterizing scattered occlusions, named CSONet. CSONet recognizes the implications of event-induced, scene-embedded, and multitudinous obstacles such as umbrellas and picket signs to achieve an accurate crowd analysis result. Finally, we developed a couple of group semantics to track and prevent crowd-caused infectious diseases. We introduce a novel COVID-19 tracing application named Crowd-based Alert and Tracing Services (CATS) and a novel face masking and social distancing monitoring system for Modeling Safety Index in Crowd (MOSAIC). CATS and MOSAIC apply privacy-aware contact tracing, social distancing, and calculate spatiotemporal Safety Index (SI) values for the individual community to provide higher privacy protection, efficient penetration of technology, greater accuracy, and effective practical policy assistance.