Automated conflation framework for integrating transportation big datasets

Abstract

The constant merging of the data, commonly known as Conflation, from various sources, has been a vital part for any phase of development, be it planning, governing the existing system or to study the effects of any intervention in the system. Conflation allows enriching the existing data by integrating information through numerous sources available out there. This process becomes unusually critical because of the complexities these diverse data bring along such as, distinct accuracies with which data has been collected, projections, diverse nomenclature adaption, etc., and hence demands special attention. Although conflation has always been a topic of interest among researchers, this area has witnessed a significant enthusiasm recently due to current advancements in the data collection methods. Even though with this escalation in interest, the developed methods didn't justify the expansions field of data collections has made. Contemporary conflation algorithms still lack an efficient automated technique; most of the existing system demands some sort of human involvement for the analysis to achieve higher accuracy. Through this work, an effort has been made to establish a fully automated process to conflate the road segments of Missouri state from two big data sources. Taking the traditional conflation a step further, this study has also focused on enriching the road segments with traffic information like delay, volume, route safety, etc., by conflating with available traffic data and crash data. The accuracy of the conflation rate achieved through this algorithm was 80-95 percent for the different data sources. The final conflated layer gives detailed information about road networks coupled with traffic parameters like delay, travel time, route safety, travel time reliability, etc.