pm storage cleanup and pm storage archive-to-swestore #34
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def cleanup(self): | ||
| for data_dir in self.app.config.get('storage', 'data_dirs'): | ||
| with filesystem.chdir(data_dir): | ||
| for run in [r for r in os.listdir(data_dir) if re.match(filesystem.RUN_RE, r)]: |
There was a problem hiding this comment.
Is this only meant for high-seq runs ? as you defined RUN_RE = '\d{6}_[a-zA-Z\d\-]+_\d{4}_[AB][A-Z\d]{9}'. So it would not work for miseq, would it ?
There was a problem hiding this comment.
damn! no it will not work on MiSeq because of the last part of the regexp, I'll have to change it, well spotted! Thanks!
|
Thanks guys, I'll fix what you reported and check the travis build on Monday, have a nice weekend! |
|
ok @senthil10 and @remiolsen , please have a second look, I've worked on your suggestions 👍 |
|
Awesome @guillermo-carrasco, looks good for me :) Hope @remiolsen will be happy with this. So merging it 👍 |
senthil10
added a commit
that referenced
this pull request
Jan 26, 2015
pm storage cleanup and pm storage archive-to-swestore
|
All good 👍 |
|
Tackar! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
pm storage cleanupwill go through alldata_dirsin the config file, check for finished runs in there (presence ofRTAComplete.txt) and move them tonosyncpm storage archive-to-swestorewill go through allnosyncdirectories within alldata_dirsin the config file, compress raw data, send it to swestore and remove it from the diskI am not double checking adler32 calculations because it is actually done, with md5sum, during the transfer to SWESTORE with the -K option.