Leveraging High Performance Computing techniques to accelerate big heterogeneous data pattern analysis and image processing
Metadata[+] Show full item record
[ACCESS RESTRICTED TO THE UNIVERSITY OF MISSOURI AT REQUEST OF AUTHOR.] In this research we discuss High Performance Computing (HPC) techniques which enable efficient and effective processing of Big Data. HPC seeks to maximize the performance of computing hardware resources. However, current trends in Big Data computing are typically using ecosystems that instead rely on layers and layers of software framework abstraction as well as increasing hardware resources (i.e., high equipment computing). We present HPC techniques which successfully utilize cur- rent multicore and heterogeneous (e.g., co-processor) computing platforms to tackle two Big Data challenges: machine learning and image processing. Machine learning in the age of Big Data requires large-scale pattern analysis to achieve a variety of computational goals. Scalability of systems designed for pattern analysis often involve complex distributed architectures, custom data structures, and distributed programming. We have developed a novel integration of graphics processing unit (GPU) hardware into the PostgreSQL (PG) DBMS to create the Data Science Engine. This achieves a scalable, cost-effective architecture for large-scale pattern analysis. Additionally, we tackle massive-scale image processing tasks (e.g., images with billions of pixels) with a multi-level parallelized geospatial data processing framework. CvTile provides the geoscience community with an open source framework that supports a wide range of geospatial data, facilitates rapid integration of novel algorithms from related image and signal processing fields. Therefore, it enables these algorithms to be scaled to meet the needs of real-world geoscience data sets by exploiting mod- ern heterogeneous HPC technologies. Both of these technologies leverage massively parallel GPU processors to successfully tackle Big Data challenges.
Access is limited to the campuses of the University of Missouri.