Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding group inputs to SaveNXcanSAS algorithm #38692

Open
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

adriazalvarez
Copy link
Contributor

@adriazalvarez adriazalvarez commented Jan 24, 2025

Description of work

This PR refactors the SaveNXcanSAS algorithm to accept group workspaces as inputs. This prepares the ground for a future algorithm for saving polarized SANS data that will need the feature. More information on : #38504

Summary of work

To save groups there are three main changes to notice:

  • The InputWorkspace property accepts a pointer to the base class Workspace that can be downcasted to a matrix workspace or group workspace. To be able to still filter appropiate compatible data and distinguish between group and matrix workspace I have added a custom LambdaValidator that does unit and compatible bins validation to each workspace in the selected input groups, that will filter appropiate groups. Additionally, it also filters compatible workspaces.
  • To handle groups in the algorithms, I have overridden the processGroups methods from the Algorithm base class, so that it will call exec when it needs to process matrix workspaces and processGroups for processing group workspaces.
  • The default behaviour for groups (although this can be discussed further) is to save each workspace of the group in an individual file and add a digit at the end of the selected filename to distinguish between files. Although we could add another input property for the algorithm with a custom suffix for group inputs.

Fixes #38504.

Further detail of work

To test:

  • Code review.
  • All tests are passing.
  • Check doc changes are appropriate.
  • Run the script and check that data is saved as intended (This uses CLI to reduce data and the training course data set):
# import mantid algorithms, numpy and matplotlib
from mantid.simpleapi import *
import matplotlib.pyplot as plt
import numpy as np
from sans.command_interface.ISISCommandInterface import *

LOQ()
MaskFile('MaskFile.toml')
AssignSample('LOQ74044.nxs')
TransmissionSample('LOQ74024.nxs', 'LOQ74014.nxs')
AssignCan('LOQ74019.nxs')
TransmissionCan('LOQ74020.nxs', 'LOQ74014.nxs')
#Performs reduction
ws_out=WavRangeReduction(2.2, 10)

ws_group=GroupWorkspaces(ws_out)

# saved in default mantid save directory

# this ought to save two separate files with suffices 00 and 01 and name test_group
SaveNXcanSAS(ws_group, 'test_group')

#this saves individual file with name test_file
SaveNXcanSAS(ws_out[0], 'test_file')

Reviewer

Please comment on the points listed below (full description).
Your comments will be used as part of the gatekeeper process, so please comment clearly on what you have checked during your review. If changes are made to the PR during the review process then your final comment will be the most important for gatekeepers. In this comment you should make it clear why any earlier review is still valid, or confirm that all requested changes have been addressed.

Code Review

  • Is the code of an acceptable quality?
  • Does the code conform to the coding standards?
  • Are the unit tests small and test the class in isolation?
  • If there is GUI work does it follow the GUI standards?
  • If there are changes in the release notes then do they describe the changes appropriately?
  • Do the release notes conform to the release notes guide?

Functional Tests

  • Do changes function as described? Add comments below that describe the tests performed?
  • Do the changes handle unexpected situations, e.g. bad input?
  • Has the relevant (user and developer) documentation been added/updated?

Does everything look good? Mark the review as Approve. A member of @mantidproject/gatekeepers will take care of it.

Gatekeeper

If you need to request changes to a PR then please add a comment and set the review status to "Request changes". This will stop the PR from showing up in the list for other gatekeepers.

@adriazalvarez adriazalvarez changed the title Addint group intpus to SaveNXcanSAS algorithm Adding group intpus to SaveNXcanSAS algorithm Jan 24, 2025
@adriazalvarez adriazalvarez changed the title Adding group intpus to SaveNXcanSAS algorithm Adding group inputs to SaveNXcanSAS algorithm Jan 24, 2025
@adriazalvarez adriazalvarez marked this pull request as ready for review January 27, 2025 08:22
@adriazalvarez adriazalvarez added this to the Release 6.13 milestone Jan 27, 2025
@adriazalvarez adriazalvarez added ISIS Team: LSS Issue and pull requests managed by the LSS subteam at ISIS SANS Issues and pull requests related to SANS labels Jan 27, 2025
@sf1919 sf1919 added the High Priority An issue or pull request that if not addressed is severe enough to postponse a release. label Jan 27, 2025
@cailafinn cailafinn self-assigned this Jan 27, 2025
@sf1919
Copy link
Contributor

sf1919 commented Jan 28, 2025

I'm removing the high priority as the cppcheck hackathon starts today. This PR may well attract conflicts so it might be best to wait until Thursday when the hackathon has finished before dealing with them.

@sf1919 sf1919 removed the High Priority An issue or pull request that if not addressed is severe enough to postponse a release. label Jan 28, 2025
@github-actions github-actions bot added the Has Conflicts Used by the bot to label pull requests that have conflicts label Jan 30, 2025
Copy link

👋 Hi, @adriazalvarez,

Conflicts have been detected against the base branch. Please rebase your branch against the base branch.

@adriazalvarez
Copy link
Contributor Author

Lots of conflicting PRs will be merged today and tomorrow, so I will wait until then to fix the conflicts.

Copy link
Contributor

@cailafinn cailafinn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple of code notes.

Working well functionally, just noticed that the files produced by the script don't have the .nxs extension. Is this the same as the original behaviour or is this an introduced change?

Framework/DataHandling/src/SaveNXcanSASBase.cpp Outdated Show resolved Hide resolved
Framework/DataHandling/src/SaveNXcanSASBase.cpp Outdated Show resolved Hide resolved
@adriazalvarez adriazalvarez force-pushed the 38504_save_groups_in_nxcansas branch 2 times, most recently from 92c833d to dd28cd1 Compare February 4, 2025 17:03
@github-actions github-actions bot removed the Has Conflicts Used by the bot to label pull requests that have conflicts label Feb 4, 2025
@adriazalvarez
Copy link
Contributor Author

@cailafinn Yes, originally the algorithm was not producing any extension when the filename property does not specify any. I have quickly added a helper function to add extension by default (.h5), but if it's not something we want here I can revert it.

Copy link
Contributor

@cailafinn cailafinn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one more note on the filepath handling.

auto const addDigit = [&](int index) {
return isGroup ? (index < 10 ? "0" + std::to_string(index) : std::to_string(index)) : "";
};
return baseFilename + addDigit(index) + ".h5";
Copy link
Contributor

@cailafinn cailafinn Feb 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like you could use the NX_CANSAS_EXTENSION you defined on L33 here.

It may also be worth editing this method and the parameter in saveSingleWorkspaceFile to handle baseFilename as a std::filesystem::path. This would allow us to use things like has_extension to only append the .h5 extension (with replace_extension) if the user hasn't supplied one already.

@adriazalvarez adriazalvarez force-pushed the 38504_save_groups_in_nxcansas branch from 7c73624 to 7ad7d98 Compare February 7, 2025 15:50
cailafinn
cailafinn previously approved these changes Feb 11, 2025
Copy link
Contributor

@cailafinn cailafinn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me now. Functionally tested using the provided script, producing files as expected. File names are correctly indexed. Code looking good.

@MialLewis This is now ready for gatekeeping when you are.

@cailafinn cailafinn requested a review from MialLewis February 11, 2025 11:10
Copy link
Contributor

@MialLewis MialLewis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks really good, thanks.

Two queries to respond to.

@@ -342,7 +335,6 @@ class SaveNXcanSASTest : public CxxTest::TestSuite {
void test_that_2D_workspace_is_saved_correctly() {
// Arrange
NXcanSASTestParameters parameters;
removeFile(parameters.filename);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a test was to error and the clean up call at the end missed, will subsequent tests fail now this has been removed?

If this could happen it could make it harder to identify and resolve any bugs revealed by the tests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I could see, when constructing the NXcanSASTestParameters struct, a new temporary file was created on the respective system temporary folder, the file was immediately removed because all that was needed for saving the NXcansas file was the path of the file. I have instead used the method Poco::TemporaryFile::tempName() on the NXcanSASTestHelper.h file to just retrieve the path of a temporary file in the folder without creating one, so there is no need to delete it just after creation.
If a test fails, there won't be a cascading effect, as the temporary names are uniquely generated and collisions would be extremely rare. If for some reason a temporary file is generated but the test fails, the file would be on a system temporary folder, and probably removed automatically by the system shortly after.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are trying to move away from Poco with File and Path. I know @jclarkeSTFC has been working on that for #37868 . I'm wondering if we should also be avoiding using Poco::TemporaryFile too.

Copy link
Contributor

@MialLewis MialLewis Feb 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://en.cppreference.com/w/cpp/io/c/tmpnam seems like a direct replacement?

I'm aware the name collision is unlikely, but as we know it is possible I'm not sure we should be introducing it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ISIS Team: LSS Issue and pull requests managed by the LSS subteam at ISIS SANS Issues and pull requests related to SANS
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Polarized SANS metadata: SaveNXcanSAS should accept workspace group inputs
5 participants