Monitoring script changes for GCP Batch [VS-1550] #9071
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mcovarr
commented
Jan 9, 2025
Comment on lines
+10
to
+17
| if [[ -e /mnt/disks/cromwell_root ]] | ||
| then | ||
| MONITOR_MOUNT_POINT_DEFAULT=/mnt/disks/cromwell_root | ||
| else | ||
| MONITOR_MOUNT_POINT_DEFAULT=/cromwell_root | ||
| fi | ||
|
|
||
| MONITOR_MOUNT_POINT=${MONITOR_MOUNT_POINT:-$MONITOR_MOUNT_POINT_DEFAULT} |
There was a problem hiding this comment.
The only changes I made in this script are here, to allow for the possibility of different mount points.
mcovarr
commented
Jan 9, 2025
| # https://support.terra.bio/hc/en-us/articles/4403215299355-Out-of-Memory-Retry | ||
| if [[ ${MEM_UNIT} == "GB" ]] | ||
| if [[ -z "${MEM_UNIT:-}" ]] |
There was a problem hiding this comment.
MEM_UNIT and MEM_SIZE are currently not being set by the GCP Batch backend so I made these changes to do something reasonable so the integration tests can run.
There was a problem hiding this comment.
We should back these changes and those below out once the issue is addressed in Cromwell.
gbggrant
approved these changes
Jan 9, 2025
| @@ -225,7 +225,7 @@ task MergeVatTSVs { | |||
| String cloud_sdk_slim_docker | |||
| } | |||
|
|
|||
| File monitoring_script = "gs://gvs_quickstart_storage/cromwell_monitoring_script.sh" | |||
| File monitoring_script = "gs://gvs_quickstart_storage/vs_1550_cromwell_monitoring_script.sh" | |||
There was a problem hiding this comment.
I presume you are going to name this something differently before this is merged?
There was a problem hiding this comment.
Yeah I'd like to discuss in post tomorrow how this thing should be named going forward.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Checking the monitoring script into git and also adapting it to GCP Batch's new mount points. We should probably discuss how to version this script for use from the GVS WDLs (e.g. a naming convention for the script file that lives in he GVS bucket or something).
There are also some changes here around the
MEM_UNITandMEM_SIZEenvironment variables that are currently not being set by Cromwell, which can be backed out once the Cromwell GCP Batch backend is updated to set them.Successful run here.