Publications

2023

1. OSTI

   Benchmarking the PCMCI Causal Discovery Algorithm for Spatiotemporal Systems

   J. Jake Nichol, Michael Weylandt, Mark Smith , and 1 more author

   2023

   Abs HTML PDF

   Causal discovery algorithms construct hypothesized causal graphs that depict causal dependencies among variables in observational data. While powerful, the accuracy of these algorithms is highly sensitive to the underlying dynamics of the system in ways that have not been fully characterized in the literature. In this report, we benchmark the PCMCI causal discovery algorithm in its application to gridded spatiotemporal systems. Effectively computing grid-level causal graphs on large grids will enable analysis of the causal impacts of transient and mobile spatial phenomena in large systems, such as the Earth’s climate. We evaluate the performance of PCMCI with a set of structural causal models, using simulated spatial vector autoregressive processes in one-and two-dimensions. We develop computational and analytical tools for characterizing these processes and their associated causal graphs. Our findings suggest that direct application of PCMCI is not suitable for the analysis of dynamical spatiotemporal gridded systems, such as climatological data, without significant preprocessing and downscaling of the data. PCMCI requires unrealistic sample sizes to achieve acceptable performance on even modestly sized problems and suffers from a notable curse of dimensionality. This work suggests that, even under generous structural assumptions, significant additional algorithmic improvements are needed before causal discovery algorithms can be reliably applied to grid-level outputs of earth system models.

2021

1. JCAM

   Machine learning feature analysis illuminates disparity between E3SM climate models and observed climate change

   J. Jake Nichol, Matthew G. Peterson, Kara J. Peterson , and 2 more authors

   Journal of Computational and Applied Mathematics, Oct 2021

   Abs HTML PDF

   In September of 2020, Arctic sea ice extent was the second-lowest on record. State of the art climate prediction uses Earth system models (ESMs), driven by systems of differential equations representing the laws of physics. Previously, these models have tended to underestimate Arctic sea ice loss. The issue is grave because accurate modeling is critical for economic, ecological, and geopolitical planning. We use machine learning techniques, including random forest regression and Gini importance, to show that the Energy Exascale Earth System Model (E3SM) relies too heavily on just one of the ten chosen climatological quantities to predict September sea ice averages. Furthermore, E3SM gives too much importance to six of those quantities when compared to observed data. Identifying the features that climate models incorrectly rely on should allow climatologists to improve prediction accuracy.

2. OSTI

   Causal Evaluations for Identifying Differences between Observations and Earth System Models

   J. Jake Nichol, Matthew Peterson, and Kara Peterson

   Oct 2021

   HTML
3. ICML

   Learning Why: Data-Driven Causal Evaluations of Climate Models.

   J. Jake Nichol, Matthew Peterson, G. Matthew Fricke , and 1 more author

   ICML 2021 Workshop Tackling Climate Change with Machine Learning, Oct 2021

   Abs HTML PDF

   We plan to use nascent data-driven causal discovery methods to find and compare causal relationships in observed data and climate model output. We will look at ten different features in the Arctic climate collected from public databases and from the Energy Exascale Earth System Model (E3SM). In identifying and comparing the resulting causal networks, we hope to find important differences between observed causal relationships and those in climate models. With these, climate modeling experts will be able to improve the coupling and parameterization of E3SM and other climate models.

2020

1. OSTI

   Arctic Tipping Points Triggering Global Change (LDRD Final Report)

   Kara J. Peterson, Amy Jo Powell, Irina Kalashnikova Tezaur , and 7 more authors

   Sep 2020

   HTML

2018

1. arXiv

   The Swarmathon: An Autonomous Swarm Robotics Competition

   Sarah M Ackerman, G Matthew Fricke, Joshua P Hecker , and 7 more authors

   Sep 2018

   Abs HTML PDF

   The Swarmathon is a swarm robotics programming challenge that engages college students from minority-serving institutions in NASA’s Journey to Mars. Teams compete by programming a group of robots to search for, pick up, and drop off resources in a collection zone. The Swarmathon produces prototypes for robot swarms that would collect resources on the surface of Mars. Robots operate completely autonomously with no global map, and each team’s algorithm must be sufficiently flexible to effectively find resources from a variety of unknown distributions. The Swarmathon includes Physical and Virtual Competitions. Physical competitors test their algorithms on robots they build at their schools; they then upload their code to run autonomously on identical robots during the three day competition in an outdoor arena at Kennedy Space Center. Virtual competitors complete an identical challenge in simulation. Participants mentor local teams to compete in a separate High School Division. In the first 2 years, over 1,100 students participated. 63% of students were from underrepresented ethnic and racial groups. Participants had significant gains in both interest and core robotic competencies that were equivalent across gender and racial groups, suggesting that the Swarmathon is effectively educating a diverse population of future roboticists.