Fujisaki-Manome, A., G. Seroka, J. Kelley, et al. (31 co-authors including Y. Liu). 2025. NOAA NOS Coastal Survey Development Laboratory, 37 pages, April 2025. NOAA Technical Memorandum NOS, 37, https://repository.library.noaa.gov/view/noaa/69898.
Executive Summary: This report documents the second round of the model evaluation effort of the National Oceanic
and Atmospheric Administration (NOAA) Unified Forecast System (UFS) Coastal Application
Team (CAT) Marine Navigation Sub-Application. The effort follows the 1) the first phase of the
project, which included gathering user requirements, generating an initial list of oceanographic
models to evaluate, and defining skill assessment guidelines for the future model evaluation
(Seroka et al. 2022), and 2) the first round of the second phase of the project– the model
evaluation (Seroka et al. 2024). The project has worked toward the major goals of UFS CAT,
which are 1) to evaluate potential coastal ocean models for the coastal ocean model
components of the UFS and 2) to train the next generation of coastal modelers to accelerate the
Research-to-Operations (R2O) process within the National Ocean Service (NOS).
Similarly to the first round of the model evaluation, two coastal ocean models were used for
evaluation: 1) Finite Volume Community Ocean Model (FVCOM); and 2) Semi-implicit Cross-
scale Hydroscience Integrated System Model (SCHISM). These two ocean models to date have
demonstrated sufficient skill to meet NOS forecast skill requirements. Therefore, specific model
results are not shown in this report; some testers’ results can be found in journal publications
that have resulted from this work. Instead, the report focuses on successes, challenges, and
lessons learned that aid future advancement of these models in NOAA Readiness Levels as
potential candidates for the UFS’ operational coastal ocean model components.
In the second round, the researchers participating in the evaluation effort (i.e., testers) refined
the model mesh for 3D simulations and incorporated atmospheric forcing in a 3D (multiple
vertical layers) baroclinic mode whereas the first round focused on 2D simulations and tidal
forcing only. The testers used atmospheric forcing from commonly used models [i.e., NOAA’s
High Resolution Rapid Refresh (HRRR), Global Forecast System (GFS), and/or the ECMWF
Reanalysis v5 (ERA5)]. In addition, the testers used output from NOAA’s Global Real-Time
Ocean Forecast System (G-RTOFS) or datasets from Copernicus Marine Environment
Monitoring Service (CMEMS) and Hybrid Coordinate Ocean Model (HYCOM) in order to create
subtidal ocean lateral boundary conditions. For river boundary conditions, testers used river
discharge observations at U.S. Geological Survey (USGS) river gauges. The model predictions
were evaluated against available water levels, currents, temperature, and salinity observations
during January 1 – March 31, 2022 and July 1 – September 30, 2021.
The key steps of this round two model evaluation include:
- Refine mesh for three-dimensional baroclinic simulations
- Conduct baseline simulations
- Compare model results with observations (water levels, water currents required; water
temperature and salinity optional) - Explore additional results to each tester’s baseline simulation
- Conduct skill assessment based on NOAA’s model evaluation guidance
Successes from the second round include continued training of the next generation of ocean
modelers, testers’ learning of the models and their exposure to NOAA’s operational processes,
skill assessment using multiple metrics, and multiple conference presentations and publications
out of the effort. Beyond improving collaboration and cooperation between modeling groups
within the government and academic partners, the UFS CAT was able to promote an evaluation
approach that uses multiple hydrodynamic models. By working together and building a coalition,
the UFS CAT team members were able to track, support, and assess new technological
advancements and algorithms pertaining to oceanographic models. Challenges include
ensuring consistency and troubleshooting support among the testers while each tester has their
own baseline, issues with Digital Elevation Models (DEMs), and the need for local knowledge to
interpret both the inputs (e.g. DEMs) and outputs (i.e. simulations results) accurately. These
successes and challenges prompted three lessons learned: 1) gap analysis (i.e. evaluating gaps
in each model’s performance and skill to bring the models to a “level playing field” for NOAA
operations), 2) realistic expectations (i.e. testers have focused so far on setting up/learning
model configurations and gaining simulation accuracy, with a lack of attention to model
efficiency, requiring co-leads to adapt and emphasize efficiency in subsequent rounds), and 3)
update to skill assessment software (i.e. further motivation for the ongoing effort of NOS’s
development of a next generation skill assessment software to have more consistency and
usability, and allow for process-based skill assessment). These lessons learned will be
implemented in the next rounds that focus on incorporating wave and hydrologic processes in
the simulations, and testing of the UFS-Coastal infrastructure.
Overall, the work with the second round helped the UFS CAT team and NOAA’s operational
coastal ocean forecasting enterprise make substantial progress toward NOAA’s central goals of
“Building a Weather Ready Nation” and “Accelerating Growth in an Information-Based Blue
Economy.”
Recent Comments