Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overrides bug #140

Merged
merged 3 commits into from
Jan 7, 2025
Merged

Overrides bug #140

merged 3 commits into from
Jan 7, 2025

Conversation

barneydobson
Copy link
Collaborator

@barneydobson barneydobson commented Jan 7, 2025

Bug found with data input overrides.

Two problems:

  • filename is the key used to specify input data at load, not data_input_dict. This created an error if applying overrides to a node with filename in the original config (as would be the case with a typical Land node).
  • Surface.apply_overrides was being called after the Land.super().apply_overrides call - thus the {"surface":overrides} were present in the super call and the No override behaviour defined for... warning was being printed.

These have been resolved by:

  • Changing data_input_dict in override keys to filename
  • Calling Surface.apply_overrides before the super call, and calling it with pop so that it wasn't present in the overrides passed to super.

@@ -440,7 +440,7 @@ def test_data_overrides(self):
node.get_data_input("temperature"),
)
# test runtime error
self.assertRaises(RuntimeError, lambda: node.apply_overrides({}))
self.assertRaises(RuntimeError, lambda: node.apply_overrides({"filename": 123}))
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@liuly12 I have changed this because I'm not sure why the previous line should have triggered an error (surely empty overrides are fine?)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, empty overrides should be fine. what error it reports?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly - before my change in the behaviour of Node.apply_overrides it would throw this error. Now it only throws the error when a non-sensible filename is passed


for surface, override in overrides.get("surfaces", {}).items():
for surface, override in overrides.pop("surfaces", {}).items():
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the fix for the warning bug

@@ -90,7 +90,7 @@ def apply_overrides(self, overrides: Dict[str, Any] = {}) -> None:
# overrides data_input_dict
from wsimod.orchestration.model import read_csv

content = overrides.pop("data_input_dict", self.data_input_dict)
content = overrides.pop("filename", None)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the fix for the data_input_dict bug

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does it only happen for Land or actually it happens in any node that requires a filename?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess any node with a data_input_dict, but since the filename override will get passed up to the Node.apply_overrides via super - it only need to be corrected here

@@ -1431,6 +1431,14 @@ def test_apply_surface_overrides(tmp_path):
import yaml
from wsimod.orchestration.model import Model
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updates in this file triggered the failing test

@barneydobson barneydobson requested a review from liuly12 January 7, 2025 09:53
@barneydobson barneydobson merged commit 7167ead into main Jan 7, 2025
20 checks passed
@barneydobson barneydobson deleted the overrides-bug branch January 7, 2025 15:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants