-
Notifications
You must be signed in to change notification settings - Fork 306
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Override runtime parameters at runtime #298
Comments
@illusional One of the many, many, many issues I have with One of the other many, many, many issues I have with this is that not everyone sees the current situation as bad :) I'd personally wholeheartedly support suggested changes to the spec which start to overhaul |
Can I suggest Maybe something like: Specifying / overriding runtime attributes from JSONWorkflow runtime attributes may additionally be specified as key/value pairs within the JSON input. The mapping from JSON or YAML values to WDL values is codified codified similar to the serialization of task inputs section. In JSON, the user may be able to specify task-specific runtime attributes similar to the following: {
"wf.t1.runtime.memory": "16 GB",
"wf.t2.runtime.cpu": 2,
"wf.t2.runtime.disks": "100 GB",
"wf.t3.runtime.arbitrary_key": ["arbitrary", "value"]
} As the runtime section consists of key/value pairs, it's up to user to ensure they provide the correct coercible type for the backend they are targeting. See the section on Type Coercion for more details. Resolution OrderSimilar to how inputs are resolved, these runtime parameters are resolved in the following order:
Edit: |
@geoffjentry I think the current situation with runtime makes it a liability in a cluster environment. |
@rhpvorderman Could you elaborate on how the current runtime is a liability? Just throwing this out there, but if runtime is going to be reworked it might also be good to consider if there are any more attributes (besides |
@illusional I like this idea. As with @rhpvorderman if there were a PR against the spec to make this modification I'd support it. @rhpvorderman I agree but echo @DavyCats in that I'd be curious to hear your specific thoughts (this might not be the right place for that, but 🤷♂️ ) @DavyCats I agree, but this is definitely not the right place for that :) Also I've learned a few times over that trying to boil the ocean on fixing runtime isn't going to work, so doing it piecemeal will get more traction. |
@DavyCats @geoffjentry
But once these issues addressed I think the runtime section will be one of the knights in shining armor defending computational biologists from mismanagement of recourses. I am very glad @illusional has addressed the first issue. |
@rhpvorderman doesn't this do what your'e describing i the second point? |
@rhpvorderman it would certainly be helpful for sizing purposes to have some sort of |
Yes. Sorry. Did not know this existed. Thanks a lot @geoffjentry ! |
Implemented by #315 |
Is there a way to override runtime parameters at runtime, except for explicitly porting them from the inputs?
eg:
Or if I did explicitly port them, does (/ should) a value of None in the runtime be override by a backend's default.
The text was updated successfully, but these errors were encountered: