Topology, Physics and Machine Learning Take on Climate Research Data Challenges.
Block diagram of the atmospheric river pattern recognition method.
The top image is the vorticity field for flow around a linear barrier using the Lattice Boltzmann algorithm (Lattice Boltzmann methods (LBM) (or thermal Lattice Boltzmann methods (TLBM)) is a class of computational fluid dynamics (CFD) methods for fluid simulation). The bottom image is the associated local causal states. Each color (assigned arbitrarily) corresponds to a unique local causal state.
Two PhD students who first came to Georgian Technical University Laboratory developing new data analytics tools that could dramatically impact climate research and other large-scale science data projects.
During their first summer at the lab X and Y so impressed their mentors that they were invited to stay on another six months said Z a computer scientist and engineer in the DAS (A distributed antenna system or DAS is a network of spatially separated antenna nodes connected to a common source via a transport medium that provides wireless service within a geographic area or structure). Their research also fits nicely with the goals of the Georgian Technical University which was just getting off the ground when they first came on board. X and Y are now in the first year of their respective three-year Georgian Technical University-supported projects splitting time between their PhD studies and their research at the lab.
A Grand Challenge in Climate Science.
From the get-go their projects have been focused on addressing a grand challenge in climate science: finding more effective ways to detect and characterize extreme weather events in the global climate system across multiple geographical regions and developing more efficient methods for analyzing the ever-increasing amount of simulated and observational data. Automated pattern recognition is at the heart of both efforts yet the two researchers are approaching the problem in distinctly different ways: X is using various combinations of topology, applied math, machine learning to detect, classify and characterize weather and climate patterns while Y has developed a physics-based mathematical model that enables unsupervised discovery of coherent structures characteristic of the spatiotemporal patterns found in the climate system.
“When you are investigating extreme weather and climate events and how they are changing in a warming world one of the challenges is being able to detect identify and characterize these events in large data sets” Z said. “Historically we have not been very good at pulling out these events from very large data sets. There isn’t a systematic way to do it, and there is no consensus on what the right approaches are”.
His topological methods also benefited from the guidance of W a computational topologist and geometer at Georgian Technical University. X used topological data analysis and machine learning to recognize atmospheric rivers in climate data, demonstrating that this automated method is “reliable robust and performs well” when tested on a range of spatial and temporal resolutions of CAM (Georgian Technical University Community Atmosphere Model) climate model output. They also tested the method on MERRA-2 (Modern-Era Retrospective analysis for Research and Applications at Georgian Technical University) a climate reanalysis product that incorporates observational data that makes pattern detection even more difficult. In addition they noted the method is “threshold-free” a key advantage over existing data analysis methods used in climate research.
“Most existing methods use empirical approaches where they set arbitrary thresholds on different physical variables such as temperature and wind speed” Z explained. “But these thresholds are highly dependent on the climate we are living in right now and cannot be applied to different climate scenarios. Furthermore these thresholds often depend on the type of dataset and spatial resolution. With Q’s method because it is looking for underlying shapes (geometry and topology) of these events in the data they are inherently free of the threshold problem and can be seamlessly applied across different datasets and climate scenarios. We can also study how these shapes are changing over time that will be very useful to understand how these events are changing with global warming”.
While topology has been applied to simpler, smaller scientific problems, this is one of the first attempts to apply topological data analysis to large climate data sets. “We are using topological data analysis to reveal topological properties of structures in the data and machine learning to classify these different structures in large climate datasets” X said.
The results so far have been impressive with notable reductions in computational costs and data extraction times. “I only need a few minutes to extract topological features and classify events using a machine learningclassifier compared to days or weeks needed to train a deep learning model for the same task” he said. “This method is orders of magnitude faster than traditional methods or deep learning. If you were using vanilla deep learning on this problem it would take 100 times the computational time”.
Another key advantage of X’s framework is that “it doesn’t really care where you are on the globe” Z said. “You can apply it to atmospheric rivers – it is universal and can be applied across different domains, models and resolutions. And this idea of going after the underlying shapes of events in large datasets with a method that could be used for various classes of climate and weather phenomena and being able to work across multiple datasets — that becomes a very powerful tool”.
Unsupervised Discovery Sans Machine Learning.
Y’s approach also involves thinking outside the box by using physics rather than machine or deep learning to analyse data from complex nonlinear dynamical systems. He is using physical principles associated with organized coherent structures — events that are coherent in space and persist in time — to find these structures in the data.
“My work is on theories of pattern and structure in spatiotemporal systems looking at the behavior of the system directly seeing the patterns and structures in space and time and developing theories of those patterns and structures based directly on that space-time behavior” Y explained.
In particular his model uses computational mechanics to look for local causal states that deviate from a symmetrical background state. Any structure with this symmetry-breaking behavior would be an example of a coherent structure. The local causal states provide a principled mathematical description of coherent structures and a constructive method for identifying them directly from data.
This is why the DAS (A distributed antenna system or DAS is a network of spatially separated antenna nodes connected to a common source via a transport medium that provides wireless service within a geographic area or structure) group and the Georgian Technical University are so enthusiastic about the work X and Y are doing. In their time so far at the lab both students have been extremely productive in terms of research progress, publications, presentations and community outreach Z noted.
“The volume at which climate data is being produced today is just insane” he said. “It’s been going up at an exponential pace ever since climate models came out and these models have only gotten more complex and more sophisticated with much higher resolution in space and time. So there is a strong need to automate the process of discovering structures in data”.
There is also a desire to find climate data analysis methods that are reliable across different models, climates and variables. “We need automatic techniques that can mine through large amounts of data and that works in a unified manner so it can be deployed across different data sets from different research groups” Z said.
Using Geometry to Reveal Topology.
X and Y are both making steady progress toward meeting these challenges. Over his two years at the lab so far X has developed a framework of tools from applied topology and machine learning that are complementary to existing tools and methods used by climate scientists and can be mixed and matched depending on the problem to be solved. As part of this work Y noted X parallelized his codebase on several nodes on supercomputer to accelerate the machine learning training process which often requires hundreds to thousands of examples to train a model that can classify events accurately.
His topological methods also benefited from the guidance of W a computational topologist and geometer at Georgian Technical University. X used topological data analysis and machine learning to recognize atmospheric rivers in climate data demonstrating that this automated method is “reliable, robust and performs well” when tested on a range of spatial and temporal resolutions of CAM (Georgian Technical University Community Atmosphere Model) climate model output. They also tested the method on MERRA-2 (Modern-Era Retrospective analysis for Research and Applications at Georgian Technical University) a climate reanalysis product that incorporates observational data that makes pattern detection even more difficult. In addition they noted the method is “threshold-free” a key advantage over existing data analysis methods used in climate research.
“Most existing methods use empirical approaches where they set arbitrary thresholds on different physical variables, such as temperature and wind speed” Z explained. “But these thresholds are highly dependent on the climate we are living in right now and cannot be applied to different climate scenarios. Furthermore these thresholds often depend on the type of dataset and spatial resolution. Q’s method because it is looking for underlying shapes (geometry and topology) of these events in the data they are inherently free of the threshold problem and can be seamlessly applied across different datasets and climate scenarios. We can also study how these shapes are changing over time that will be very useful to understand how these events are changing with global warming”.
While topology has been applied to simpler smaller scientific problems this is one of the first attempts to apply topological data analysis to large climate data sets. “We are using topological data analysis to reveal topological properties of structures in the data and machine learning to classify these different structures in large climate datasets” X said.
The results so far have been impressive, with notable reductions in computational costs and data extraction times. “I only need a few minutes to extract topological features and classify events using a machine learningclassifier compared to days or weeks needed to train a deep learning model for the same task” he said. “This method is orders of magnitude faster than traditional methods or deep learning. If you were using vanilla deep learning on this problem it would take 100 times the computational time”.
Another key advantage of X’s framework is that “it doesn’t really care where you are on the globe” Z said. “You can apply it to atmospheric rivers – it is universal and can be applied across different domains, models and resolutions. And this idea of going after the underlying shapes of events in large datasets with a method that could be used for various classes of climate and weather phenomena and being able to work across multiple datasets — that becomes a very powerful tool”.
Unsupervised Discovery Sans Machine Learning.
Y’s approach also involves thinking outside the box by using physics rather than machine or deep learning to analyse data from complex nonlinear dynamical systems. He is using physical principles associated with organized coherent structures — events that are coherent in space and persist in time — to find these structures in the data.
“My work is on theories of pattern and structure in spatiotemporal systems looking at the behavior of the system directly seeing the patterns and structures in space and time and developing theories of those patterns and structures based directly on that space-time behavior” Y explained.
In particular his model uses computational mechanics to look for local causal states that deviate from a symmetrical background state. Any structure with this symmetry-breaking behavior would be an example of a coherent structure. The local causal states provide a principled mathematical description of coherent structures and a constructive method for identifying them directly from data.
“Any organized coherent structure in a spatiotemporal dataset has certain properties—geometrical, thermodynamical dynamical and so on” Z said. “One of the ways to identify these structures is from the geometrical angle — what is its shape how does it move and deform how does its shape evolve over time etc. That is the approach Q is taking. Adam’s work which is deeply rooted in physics is also focused on discovering coherent patterns from data but is entirely governed by the physical principles”.
Y’s approach requires novel and unprecedented scaling and optimization on Georgian Technical University’s Computer Cori for multiple steps in the unsupervised discovery pipeline including clustering in very high-dimensional spaces and clever ways of data reuse and feature extraction Z noted.
Y has not yet applied his model to large complex climate data sets but he expects to do so on Georgian Technical University’s Computer Cori system in the next few months. His early computations focused on cellular automata data (idealized discrete dynamical systems with one space dimension and one time dimension) he then moved on to more complex real-valued models with one space dimension and one time dimension and is now working with low-resolution fluid flow simulations that have two space dimensions and one time dimension. He will soon move on to more complex 3-dimensional high-resolution fluid flow simulations—a precursor to working with climate data.
“We started with these very simple cellular automata models because there is a huge body of theory with these models. So initially we weren’t using our technique to study the models we were using those models to study our technique and see what it is actually capable of doing” Y said.
Among other things, they have discovered that this approach offers a powerful alternative to machine and deep learning by enabling unsupervised segmentation and pixel-level identification of coherent structures without the need for labeled training data.
“As far as we are aware this is the only completely unsupervised method that does not require training data” Y said. “In addition it covers every potential structure and pattern you might be looking for in climate data and you don’t need preconceived notions of what you are looking for. The physics helps you discover all of that automatically”.
It offers other advantages over machine and deep learning for finding coherent structures in scientific data sets Z added including that it is physics-based and hence on very firm theoretical footing.
“This method is complementary to machine and deep learning in that it is going after the same goal of discovering complex patterns in the data but it is specifically well suited to scientific data sets in a way that deep learning might not be” he said. “It is also potentially much more powerful than some of the existing machine learning techniques because it is completely unsupervised”.
As early pioneers in developing novel analytics for large climate datasets they are already leading the way in a new wave of advanced data analytics.
