[Internal] Updating runtime parameters

[sage-wright]

This one is a doozy.

This PR:

• closes Change version_capture container #483
• closes shovill-pe CPUs not set in task #485
• closes ksnp3 doesn't have CPUs passed in #487
• closes Check RAM Midas uses  #488
• closes Reduce Gambit task resource requirements #489
• closes Check cg-pipeline CPUs used #490
• closes Update check-reads to use less CPUs #491
• closes Make busco preemptible #492
• closes Make MIDAS optional in TheiaProk #493

🗑️ This dev branch should be deleted after merging to main.

🧠 Aim, Context and Functionality

Many tasks require far more runtime parameters than are actually used. This PR adjusts various runtime parameters accordingly.

🛠️ Impacted Workflows/Tasks & Changes Being Made

This will affect the behavior of the workflow(s) even if users don’t change any workflow inputs relative to the last version : No

Running this workflow on different occasions could result in different results, e.g. due to use of a live database, "latest" docker image, or stochastic data processing : No

• No change in workflow function, just optimization of resources

📋 Workflow/Task Step Changes

🔄 Data Processing

Compute resources changed:

• many, see also code changes

➡️ Inputs

None

⬅️ Outputs

None

🧪 Testing

Terra Testing

Suggested Scenarios for Reviewer to Test

Ensure costs and runtime attributes went down as expected.

Theiagen Version Release Testing (optional)

None

🔬 Final Developer Checklist

• The workflow/task has been tested locally and results, including file contents, are as anticipated
• The workflow/task has been tested on Terra and results, including file contents, are as anticipated
• The CI/CD has been adjusted and tests are passing (to be completed by Theiagen developer)
• Code changes follow the style guide

🎯 Reviewer Checklist

• All impacted workflows/tasks have been tested on Terra with a different dataset than used for development
• All reviewer-suggested scenarios have been tested and any additional
• All changed results have been confirmed to be accurate
• All workflows/tasks impacted by change/s have been tested using a standard validation dataset to ensure no unintended change of functionality
• All code adheres to the style guide
• MD5 sums have been updated
• The PR author has addressed all comments

🗂️ Associated Documentation (to be completed by Theiagen developer)

• Relevant documentation on the Public Health Resources "PHB Main" has been updated
• Workflow diagrams have been updated to reflect changes

[AndrewLangvt]

TheiaProk_Illumina_PE_PHB - https://app.terra.bio/#workspaces/theiagen-training-workspaces/Theiagen_Lang_Sandbox/job_history/4fa24d88-2417-4fa3-a3e1-3f4268ca1198
shovil - confirmed CPUs are passed to shovil command
busco - confirmed preemtible
gambit - confirmed disk size 20, confirmed mem 2, CPU 1

realized I had to pass in flag to run midas- https://app.terra.bio/#workspaces/theiagen-training-workspaces/Theiagen_Lang_Sandbox/job_history/1fa8a187-17f8-4638-9ba3-965adb8e1c73
midas - confirmed preemtible, mem = 4

@jrotieno from my perspective, once you review your set of these workflows, you can mark it approved for merge

[jrotieno]

For the additional TheiaProk tasks assigned to me, I just used Andrew Lang's runs:
check-reads:
2GB RAM, 100GB Disk, 1 CPU and preemptible TRUE
cg-pipeline:
2GB RAM, 50GB Disk, 1 CPU and preemptible TRUE
version_capture
container used is the alpine-plus-bash. Not seen the version though (3.20.0), should I have in the compute details?
1GB RAM, 10GB Disk, 1 CPU and preemptible TRUE

kSNP3 CPUs:
https://job-manager.dsde-prod.broadinstitute.org/jobs/b7e86f67-8764-49d4-8456-b0d3d7f7944a
4GB RAM, 100GB Disk, 2 CPU and preemptible FALSE

[jrotieno]

@AndrewLangvt maybe you just need to mark the ones you went through as done on the board?

[AndrewLangvt]

@jrotieno good call; done!

[jrotieno]

@AndrewLangvt and I have reviewed the changes and approve. Well done @sage-wright