Processing Spatio-temporal Data On Map-Reduce

Abstract

The amount of spatio-temporal data generated in numerous scientific and industrial settings have exploded in recent years. Without a distributed platform, supporting efficient analytics operations over such voluminous datasets become prohibitively expensive. As a result there has been an increasing interest in using map-reduce to parallelize the processing of large-scale spatio-temporal data. While Hadoop, which has become the de-facto implementation of map-reduce, has shown to be effective in handling large volumes of unstructured data, several key issues needs to be addressed to exploit its power for processing spatio-temporal data.

In this tutorial, we explore design techniques for spatio-temporal analytics and data management on Hadoop, based on recent work in this area. We outline strategies for devising map-reduce algorithms for performing fundamental spatial analytics involving computational geometry operations as well as two-way and multi-way spatial join operations. We discuss storage optimization techniques such as chunking and colocation to enable efficient organization of multi-dimensional data on HDFS along with indexing techniques for fast spatial data access.