Blog

Abstract

The COAL poster abstract was accepted into the IN31B: Exploiting Big Earth Data: GIS and Beyond poster session of the Earth and Space Science Informatics meeting section.

Mining is known to cause environmental degradation, but software tools to identify its impacts are lacking. However, remote sensing, spectral reflectance, and geographic data are readily available, and high-performance cloud computing resources exist for scientific research. Coal and Open-pit surface mining impacts on American Lands (COAL) provides a suite of algorithms and documentation to leverage these data and resources to identify evidence of mining and correlate it with environmental impacts over time.

COAL was originally developed as a 2016 – 2017 senior capstone collaboration between scientists at the NASA Jet Propulsion Laboratory (JPL) and computer science students at Oregon State University (OSU). The COAL team implemented a free and open-source software library called "pycoal" in the Python programming language which facilitated a case study of the effects of coal mining on water resources. Evidence of acid mine drainage associated with an open-pit coal mine in New Mexico was derived by correlating imaging spectrometer data from the JPL Airborne Visible/InfraRed Imaging Spectrometer – Next Generation (AVIRIS-NG), spectral reflectance data published by the USGS Spectroscopy Laboratory in the USGS Digital Spectral Library 06, and GIS hydrography data published by the USGS National Geospatial Program in The National Map. This case study indicated that the spectral and geospatial algorithms developed by COAL can be used successfully to analyze the environmental impacts of mining activities.

Continued development of COAL has been promoted by a Startup allocation award of high-performance computing resources from the Extreme Science and Engineering Discovery Environment (XSEDE). These resources allow the team to undertake further benchmarking, evaluation, and experimentation using multiple XSEDE resources. The opportunity to use computational infrastructure of this caliber will further enable the development of a science gateway to continue foundational COAL research.

This work documents the original design and development of COAL and provides insight into continuing research efforts which have potential applications beyond the project to environmental data science and other fields.

Presentation

COAL was presented at the Fall Meeting on Wednesday morning (2017-12-13 08:00-12:20) at the Ernest N. Morial Convention Center in New Orleans, Louisiana. The COAL poster was well-received, and many gainful conversations were shared with international scientists, regulators, and industry representatives with stakes in remote sensing and environmental monitoring. Valuable feedback and insight was provided by experienced geoscience researchers and developers. The opportunity to showcase COAL research, to take in the diversity of science on display, and to savor New Orleans culture and cuisine made this a very worthwhile experience.

Poster

The COAL conference poster was drafted as a formal conclusion of the first stage of research and development. The large format made it possible to feature a high-quality print of the mine classification imagery from the initial case study which shows off small details and their geographical context. Additional figures enhanced interpretation, provided unique visualizations, and highlighted COAL data products. The accompanying text described the context, methodology, conclusions, acknowledgements, and references of COAL research and development. Preview the poster below, view the source and release on GitHub, or access the publication on Zenodo.

COAL was presented to attendees of the 2017 HyspIRI Science and Applications Workshop which took place at the California Institute of Technology, Beckman Institute Auditorium, 1200 E California Blvd Pasadena, CA 91125 from Oct 17th - 19th. COAL was presented alongside a huge variety of outstanding work in the area of Global Earth Imaging Spectroscopy and Thermal Infrared Measurements.

The workshop was focused on everything related to the NASA Decadal Survey Mission Concept named HyspIRI; Hyperspectral Infrared Imager. HyspIRI will study the world’s ecosystems and provide critical information on natural disasters such as volcanoes, wildfires and drought. HyspIRI will be able to identify the type of vegetation that is present and whether the vegetation is healthy. The mission will provide a benchmark on the state of the worlds ecosystems against which future changes can be assessed. The mission will also assess the pre-eruptive behavior of volcanoes and the likelihood of future eruptions as well as the carbon and other gases released from wildfires.

The HyspIRI mission includes two instruments mounted on a satellite in Low Earth Orbit. There is an imaging spectrometer measuring from the visible to short wave infrared (VSWIR: 380 nm - 2500 nm) in 10 nm contiguous bands and a multispectral imager measuring from 3 to 12 um in the mid and thermal infrared (TIR). The VSWIR and TIR instruments both have a spatial resolution of 60 m at nadir. The VSWIR will have a revisit of of 19 days and the TIR will have a revisit of 5 days. HyspIRI also includes an Intelligent Payload Module (IPM) which will enable direct broadcast of a subset of the data.

The data from HyspIRI will be used for a wide variety of studies primarily in the Carbon Cycle and Ecosystem and Earth Surface and Interior focus areas. The mission was recommended in the 2007 National Research Council Decadal Survey requested by NASA, NOAA, and USGS.The mission is currently at the study stage and this website is being provided as a focal point for information on the mission and to keep the community informed on the mission activities.

We intend to extend COAL to accomodate data from HyspIRI once it is available. In the meantime we look forward to continued work and collaborations with members of the Global Earth Imaging Spectroscopy and Thermal Infrared Measurements community. The COAL poster presented at the HyspIRI Workshop can be downloaded from our publications page.

We we absolutely stoked to receive excellent news that the COAL Follow-On (COAL-FO) project proposal was accepted by Oregon State Universities Senior Capstone program. This will enable us to continue critical work into COAL software development, research and utilization of the XSEDE Startup resources we have available.

We are very happy to welcome Kenneth Bertino Thompson and Bryce Egley, our new Senior Computer Scientists from OSU's College for Electrical Engineering and Computer Science to the COAL team. The 2017-2018 project is the successor project to the 2016-2017 COAL project. COAL initially aimed to deliver a suite of algorithms to identify, classify, characterize, and quantify (by reporting a number of key metrics) the direct and indirect impacts of mining operations and related destructive surface mining activities across the continental U.S (and further afield). COAL successfully delivered a Python library for processing hyperspectral imagery from remote sensing devices such as the Airborne Visible/InfraRed Imaging Spectrometer (AVIRIS) and a Science Data System for running COAL pipelines. COAL-FO will utilize recent funding obtained from a recently awarded NSF-funded XSEDE high performance computing (HPC) grant to further improve, validate and document COAL algorithms, execution runtime performance and geospatial output results.

The COAL final report with code and digital artifacts was submitted at the end of the term, completing the 2016 – 2017 senior capstone project. The hard copy of the printed report is on file with the OSU School of Electrical Engineering and Computer Science where it may be shared with students and industry sponsors. This blog post describes how to obtain digital copies of the report and associated files.

The entire COAL final report submission including all documents, code, and digital artifacts is hosted on Box for Oregon State University. The archive is in tar.bz2 format and is 8.8GB in size. Enclosed within the archive is a README which summarizes the locations of important resources such as documents, presentations, and requisite files.

The San Juan Mine case study QGIS project and files can be downloaded separately from Google Drive. The archive is in tar.gz format and is 2GB in size (7GB uncompressed). It contains visible-light, mining classified, and environmental correlation imagery as well as mine permit, hydrography, elevation, and transportation data. Extract the archive and open the project file san‑juan‑mine.qgs with a build of QGIS linked against GDAL 2.2.0+.

Contact the COAL team with any questions about accessing the data.

The COAL team was honored to be recognized on the official blog of the School of Electrical Engineering and Computer Science at Oregon State University for its achievements during the 2016-2017 senior software engineering project. The post documents the team's past accomplishments and future work. Read more at OSU EECS News.

COAL was recently awarded an XSEDE Startup Allocation which will be used to continue application development by the team. In addition, we will be looking to undertake benchmarking, evaluation and experimentation on the various XSEDE resources. The availability and opportunity to use computational infrastructure of this calibre will further enable the development of a science gateway to continue foundational COAL research.

Specifically the allocation details are as follows

• PI: Lewis McGibbney, NASA JPL
• Co-I's: Taylor Alexander Brown, Heidi Ann Clayton and Xiaomei Wang of OSU
• Request: Coal and Open-pit surface mining impacts on American Lands (COAL)
• Request Number: EAR170010 (New)
• Start Date: N/A
• End Date: 2018-05-21
• Awarded Resources: PSC Large Memory (Bridges Large): 1,000.0 SUs PSC Storage (Bridges Pylon): 2,000.0 GB

The estimated value of these awarded resources is $732.00. The allocation of these resources represents a considerable investment by the NSF in advanced computing infrastructure for U.S. The dollar value of the COAL allocation is estimated from the NSF awards supporting the allocated resources.

Generally speaking we are extermely happy to be in receipt of such an award. It is our aim and intention to publish the progress we make in a series of conference, workshop and journal activities... so watch this space!

We are proud to announce the immediate release and availability of COAL 0.5.2. We urge all users of the pycoal Python package to upgrade to this release immediately.

What is COAL?

COAL is a Python library for processing hyperspectral imagery from remote sensing devices such as the Airborne Visible/InfraRed Imaging Spectrometer (AVIRIS). COAL was originally developed as a 2016 – 2017 senior capstone collaboration between scientists at the Jet Propulsion Laboratory (JPL) and computer science students at Oregon State University (OSU). COAL aims to provide a suite of algorithms for classifying land cover, identifying mines and other geographic features, and correlating them with environmental data sets. COAL is Free and Open Source Software with the pycoal toolkit licensed under the terms of the GPL v2.0.

How do I install it?

The Python COAL package pycoal can be installed from the cheeseshop

$ pip install pycoal

or from conda

$ conda install -c conda-forge pycoal

How do I get started?

See our Quickstart Documentation.

Community and Development

To become involved or if you require help using the project request to join our mailing list. If you have issue using COAL, please log a ticket in our Github issue tracker.

Taylor, Heidi, and Xiaomei from the COAL team presented the poster and demo at the Undergraduate Engineering Expo 2017 on Friday, May 19^th 2017. Expo featured more than 200 projects by more than 600 undergraduate students from the College of Engineering at Oregon State University. COAL was promoted by Expo media as an example of computer science excellence. The COAL team shared its methodology and findings with members of the academic community, representatives from industry, and the general public and enjoyed in-depth conversations about remote sensing, environmental science, and software development.

COAL was selected for one of four industry awards presented during Expo, the CH2M Multidisciplinary Collaboration Award. The COAL team spoke with CH2M representatives who were interested in its work with the scientific and Free Software communities and the possible applications of its remote sensing techniques to environmental remediation efforts. After several rounds of interviews with the judges, COAL was chosen based on the following criteria:

Complex projects rarely exist within a single technical field. Collaboration between technical specialists is necessary to meet client needs and design criteria. Quality begins with a defined design development process. CH2M will award the Multidisciplinary Collaboration Award to a project team that clearly demonstrates proactive planning and collaboration with others at key points throughout project execution. The impact of the collaboration should be to produce an innovative final product which returns high value to the project sponsor/client or to overcome unique technical challenges. The selected team shall be prepared to describe the design process leading up to the final product, and the lessons learned along the way.

The award was presented during the ceremony at the end of Expo. The COAL team was honored to be recognized by OSU and CH2M.

The presentation at Expo provided opportunities to promote the COAL project and communicate its methodology and findings to the public. The interest from members of the community demonstrated that remote sensing is an increasingly important technique for environmental science. The COAL team is pleased with a successful conclusion to its senior capstone collaboration between OSU and JPL.