When Hurricanes Harvey (2017) and Florence (2018) hit, it was not solely the storm surge from the Gulf of Mexico and Atlantic Ocean that led to flooding. Inland sources, like rain-swollen rivers, lakes, and suburban culverts also contributed significantly. These factors were missed by many computer models at the time, which underestimated the flood risk.
"People don't care as much as to whether flooding is coming from the river or the ocean, especially when both contribute to water levels, as they want to know, ‘Is my house going to be flooded?'" said Edward Myers, branch chief of the Coastal Marine Modeling Branch, located in the Coast Survey Development Laboratory at the National Oceanographic and Atmospheric Administration (NOAA).
Myers and his colleagues at NOAA are collaborating with Y. Joseph Zhang from the Virginia Institute of Marine Science (VIMS) at William & Mary to develop and test the world's first three-dimensional operational storm surge model.
"We started with the right attitude and the right core algorithm," joked Zhang, research professor at the Center for Coastal Resources Management. "Over the years, we've re-engineered the dynamic core multiple times and that led to the current modeling system."
"The project is a great demonstration of how, when we all work together — the academic community, government agencies, and coastal communities — wonderful things happen."
Now in its third incarnation, the Semi-implicit Cross-scale Hydroscience Integrated System Model (SCHISM) forecasts coastal flooding in Taiwan, at agencies across the European Union, and elsewhere. It is being considered for operational use by NOAA. (The researchers described the system in the Nov. 2021 issue of EOS, the science news magazine of the American Geophysical Union.)
SCHISM is designed to serve the needs of a wide range of potential users. "Compound surge and flooding is a world-wide hazard," Zhang said. "It's notoriously challenging, especially in the transition zone where the river meets the sea. Lots of factors come into play and interact non-linearly."
Two different means of assessing the nonlinear influence of different effects (from the ocean and atmosphere, rivers, and precipitation) on compound flooding are shown here. (a) Compound flood factors in the Houston area during Hurricane Harvey; (b) Individual flood drivers were dominantly responsible for the vast majority of flooding in some regions (blue, yellow, red) along the coast of the Carolinas during Hurricane Florence. Credit: Wei Huang; base maps in (a) and (b): Google Maps
Surrounding the hydrodynamic core of SCHISM are numerous modules that simulate other phenomena important to flooding. These include air-sea exchange, vegetation, and sediment. Other modules adapt the system for specific events, like oil spills, or to predict conditions, like water quality.
Unlike other coupled inland-coastal flooding models, SCHISM incorporates very fine scale features, like engineered structures, culverts, and narrow gates, into its forecasts.
"We see the SCHISM model as a candidate for what we want to do in terms of linking models," Myers said. "It handles very fine scales very well, and we can run it efficiently. Of course, it still requires high performance computing, which is why the Texas Advanced Computing Center [TACC] has been instrumental."
Zhang started using TACC systems in 2010. "Since then, TACC's HPC resources and facilities have greatly improved. The size of the problem we can tackle has dramatically improved. We've gotten into uncharted territory," Zhang said. "TACC has been continuously pushing the limits."
Verifying the model
To confirm the accuracy of SCHISM, Zhang has been focusing on hindcasts — a way of testing a mathematical model by inputting estimates from past events and seeing how well the model output matches the known results.
Writing in Natural Hazards and Earth System Sciences in June 2021, Zhang, Myers, and their collaborators described a study of compound flooding during Hurricane Florence that impacted a large swath of North Carolina in September 2018. The study found that barrier islands played an important role in compound surges.
"Barrier islands somehow can amplify or dampen surge, depending on what direction the surge is coming from," Zhang said. They also found that the wave effects are significant near the barrier islands and have contributed to some observed over-toppings and breaches.
Applying each of the three major forcing factors — oceanic, fluvial (from rivers), and pluvial (from surface flooding) — separately, they were able to produce a "dominance map" that shows which factors are most important for a specific site. The map clearly demonstrated significant compound effects in most of the affected coastal watersheds, estuaries, and back bays behind the barrier islands.
Creek-to-ocean 3D model results including NWM streamflow and precipitation for predicting the impact of Hurricane Irene (2011) zoomed in Trenton, NJ region. A) Model domain. Circles show Hurricane Irene (2011) track. B) High resolution mesh on the Delaware Bay region. C) Total water level above the ground. D) Extra inundation (up to 2 m) due to compound inland-coastal and freshwater flooding. [Credit: Moghimi, Myers, Pe'eri, Zhang and Ye]
The model's ability to resolve small-scale features in some watershed areas turned out to be critical for capturing the observed high-water marks locally.
The research was enabled by William & Mary Research Computing, the Extreme Science and Engineering Discovery Environment (XSEDE), which allocated time on TACC's Stampede2 system, and the NASA High-End Computing (HEC) program through the NASA Advanced Supercomputing (NAS) Division at Ames Research Center.
Real-time flood forecasting
The NOAA and VIMS team started their interaction with TACC after NOAA Storm Surge Modeling Team lead Saeed Moghimi connected with individuals from DesignSafe, an NSF program for natural hazards, that relies on advanced computing capabilities at TACC. The center initially provided a small allocation to develop and test the first SCHISM prototype on TACC systems.
That relationship quickly blossomed. Since April 2021, NOAA has run daily 2D and 3D compound flood models for the East and Gulf Coasts on Frontera, the fastest academic supercomputer in the world and the 13th fastest overall.
Early indications suggest the Inland-Coastal Flooding Operational Guidance System is working well. During Tropical Storm Claudette (June 2021), the researchers compared water level results between the three-dimensional operational storm surge model and NOAA's current model.
"We were quite pleased with the results," Zhang said. "They corroborated with inundation observations at Shoreline Park, Mississippi, and in Slidell, Louisiana, and the forecasts were all done in real-time."
Myers concurred. "We're really impressed with how the model is performing in terms of its accuracy, computational efficiency, and robustness. For NOAA to put a model into operations, it needs to be very reliable. SCHISM has shown itself to be extremely robust."
"Having this test going a full year to make sure everything is going well is hugely important," said Moghimi. "Even if we put this version of the system into production, having a shadow application of what we have at NOAA would be extremely valuable and provide an opportunity for agile upgrading of the system in the future."
An urgent computing hub
The flood guidance project dovetails with a growing area of expertise at TACC: providing computing resources to support urgent computing.
TACC has contributed to such efforts since 2003, when the center's supercomputers were used to guide investigators to debris from the Space Shuttle Columbia in East Texas. During Hurricanes Ike and Harvey, the Deep Water Horizon oil spill, and earthquake recovery efforts in Haiti and Japan, TACC provided emergency computing resources. In the early phases of the pandemic, TACC devoted more than 30% of its computing resources to COVID-19 research.
The daily 3D coastal flooding now-cast and two-day forecasts require approximately 2,200 computing cores for 50 minutes; the 2D forecasts use 1,000 cores for 15 minutes. While this scale of computing is not huge, compared to many other Frontera projects, it would be difficult to do consistently, for months, anywhere else.
Powering this type of project became a bit easier when, in 2021, with support from NSF and Dell EMC, TACC expanded Frontera to support urgent computing, pandemic research, and rapid responses to natural and man-made disasters.
"The NOAA project is a great example of how TACC resources can be used to improve our response to natural hazards," said Tim Cockerill, TACC user services director, "both by advancing basic research into compound flooding, and by providing real-time forecasts in times of disaster."
The system represents a way to provide better information to the public and decision-makers in times of crisis. Zhang stated, "The project is a great demonstration of how, when we all work together — the academic community, government agencies, and coastal communities — wonderful things happen."