Fix NHD issues related to gage byte strings

[shorvath-noaa]

There was inconsistent handling of converting USGS gage IDs from byte strings to character strings. This resulted in link-gage-crosswalk and lake-gage-crosswalk having different character representations of the same gage IDs. The position where _reindex_link_to_lake_id() function was also caused issues as it replaced link IDs with lake IDs in usgs_df before creating reservoir_usgs_df. This resulted in missing lake IDs/gage IDs in reservoir_usgs_df.

Additions

Removals

Changes

DataAssimilation.py

• Moved location of _reindex_link_to_lake_id()

NHDNetwork.py

• Updated accessing reservoir_da parameters to conform to config-module. This incorporates changes from PR update on NHD to read reservoir_persistence_usgs #737

nhd_io.py

• Convert gage ID byte strings to character strings in a consistent way.

nhd_network.py

• Convert gage ID byte strings to character strings in a consistent way.

preprocess.py

• Moved location of _reindex_link_to_lake_id()

Testing

1. Ran nwm_routing v3 and v4 on NHD network with extended_AnA configuration (1-day run).

Screenshots

Notes

Todos

Checklist

• PR has an informative and human-readable title
• Changes are limited to a single goal (no scope creep)
• Code can be automatically merged (no conflicts)
• Code follows project standards (link if applicable)
• Passes all existing automated tests
• Any change in functionality is tested
• New functions are documented (with a description, list of inputs, and expected output)
• Placeholder code is flagged / future todos are captured in comments
• Visually tested in supported browsers and devices (see checklist below 👇)
• Project documentation has been updated (including the "Unreleased" section of the CHANGELOG)
• Reviewers requested with the Reviewers tool ➡️

Testing checklist

Target Environment support

• Windows
• Linux
• Browser

Accessibility

• Keyboard friendly
• Screen reader friendly

Other

• Is useable without CSS
• Is useable without JS
• Flexible from small to large screens
• No linting errors or warnings
• JavaScript tests are passing

[AminTorabi-NOAA]

I got an error in compute.py that says KeyError: "None of [Index([5194634], dtype='int32', name='lake_id')] are in the [index]"
seems like this line below and previous change on L139 won't run because from_file is True
self._usgs_df = _reindex_link_to_lake_id(self._usgs_df, network.link_lake_crosswalk)

Some general improvement : in compute.py it happens that usgs_wbodies_sub is empty so reservoir_usgs_df.loc[usgs_wbodies_sub] would be empty there is no need to calculate those. We should skip the empty ones

[shorvath-noaa]

I'm not sure where in the code this comment is referring to. Can you make comments specific to parts of the code?

…

On Mon, Feb 26, 2024 at 11:20 AM Amin Torabi ***@***.***> wrote: I got an error in compute.py that says KeyError: "None of [Index([5194634], dtype='int32', name='lake_id')] are in the [index]" seems like this line below and previous change on L139 won't run because from_file is True self._usgs_df = _reindex_link_to_lake_id(self._usgs_df, network.link_lake_crosswalk) *Some general improvement :* in compute.py it happens that usgs_wbodies_sub is empty so reservoir_usgs_df.loc[usgs_wbodies_sub] would be empty there is no need to calculate those. We should skip the empty ones — Reply to this email directly, view it on GitHub <#742 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AYSH2PPD2RTPUV6EQZMEQETYVTG6JAVCNFSM6AAAAABDXB2C5KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRUHAZDSMBTGM> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[AminTorabi-NOAA]

This part of the code is not running while from_files (L295) is True

[AminTorabi-NOAA]

This part of the code is not running while from_files (L100) is True