Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added more robust archive file corruption handling #370

Merged
merged 1 commit into from
Jun 16, 2023

Conversation

eagoetz
Copy link
Collaborator

@eagoetz eagoetz commented Jun 12, 2023

This PR will add an additional check when reading archive hdf5 files so that each group is read once to check that there are no runtime errors due to file corruption in the group

@eagoetz eagoetz marked this pull request as draft June 12, 2023 22:03
@eagoetz eagoetz self-assigned this Jun 12, 2023
@codecov
Copy link

codecov bot commented Jun 12, 2023

Codecov Report

Merging #370 (ca96e71) into master (7a3f4d3) will decrease coverage by 0.02%.
The diff coverage is 33.33%.

❗ Current head ca96e71 differs from pull request most recent head 255b623. Consider uploading reports for the commit 255b623 to get more accurate results

@@            Coverage Diff             @@
##           master     #370      +/-   ##
==========================================
- Coverage   50.29%   50.28%   -0.02%     
==========================================
  Files          60       60              
  Lines        8669     8678       +9     
==========================================
+ Hits         4360     4363       +3     
- Misses       4309     4315       +6     
Flag Coverage Δ
Linux 50.28% <33.33%> (-0.02%) ⬇️
python3.10 50.28% <33.33%> (-0.02%) ⬇️
python3.7 49.47% <33.33%> (-0.02%) ⬇️
python3.8 50.28% <33.33%> (-0.02%) ⬇️
python3.9 50.28% <33.33%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
gwsumm/archive.py 71.94% <33.33%> (-1.86%) ⬇️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Collaborator

@areeda areeda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@eagoetz
Copy link
Collaborator Author

eagoetz commented Jun 13, 2023

@areeda Thanks, I am working on this with @iaraota shadowing. We are still testing out a solution. I don't think we've reached a mature state just yet. Stay tuned to this space!

This PR will add an additional check when reading archive hdf5 files so that each group is read once to check that there are no runtime errors due to file corruption in the group
@eagoetz eagoetz requested a review from areeda June 15, 2023 16:08
@eagoetz eagoetz added this to the 2.1.6 milestone Jun 15, 2023
Copy link
Collaborator

@areeda areeda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good

@eagoetz eagoetz marked this pull request as ready for review June 16, 2023 17:06
@eagoetz eagoetz merged commit 77f9a9a into gwpy:master Jun 16, 2023
@eagoetz eagoetz deleted the rm-hdf5 branch June 16, 2023 17:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants