Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check cg-pipeline CPUs used #490

Closed
andrewjpage opened this issue May 24, 2024 · 0 comments · Fixed by #494
Closed

Check cg-pipeline CPUs used #490

andrewjpage opened this issue May 24, 2024 · 0 comments · Fixed by #494
Assignees

Comments

@andrewjpage
Copy link
Contributor

andrewjpage commented May 24, 2024

🐛

📝 Describe the Issue

The cg-pipeline task doesnt take CPUs as input. 4 are passed in. Does it use all available?

Pass in the set number of CPUs in the task.
Run the task and figure out how much RAM is normally used (/usr/bin/time -v cmd).
Does it really need 8GB?
Does it require a 100GB local disk?
How much processing time is used and calculate the utilisation of the CPUs? If its not making use of all 4, adjust the task to use less.

This task is very short. Make it preemptible (spot) so that we can access lower pricing. Google give 30 seconds notice before killing it, so we shouldn't notice any difference.

In the runtime section of the task set:

maxRetries: 3
preemptible: 1
@andrewjpage andrewjpage changed the title cg-pipeline Check cg-pipeline CPUs used May 24, 2024
@sage-wright sage-wright self-assigned this May 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants