Big Data Meets Geo-Computation: Combining Research Reproducibility and Processing Efficiency at Yale High Performance Computing

Yale School of Management

In recent years there has been an explosion of geo-datasets derived from an increasing number of remote sensors, field instruments, sensor networks, and other GPS-equipped "smart" devices. Big Data processing requires flexible tools that combine efficient processing, either on your local pc or on remote servers (e.g, clusters - HPCs). However, leveraging these new data streams requires new tools and increasingly complex workflows often involving multiple software and/or programming languages. This also the case for GIS and Remote Sensing analysis where statistical/mathematical algorithms are implemented in complex geospatial workflows. I will show few examples of environmental applications where I combine different open-source geo-libraries for a massive computation at YALE High-performance computing. Moreover, I will show the ongoing activities at YCRC in organizing workshops to enlarge the GeoComputation knowledge of the Yale Community