Restructure suite run time log/job/ and work/ #1069
Conversation
|
@hjoliver please take an initial look. |
| @@ -257,7 +257,7 @@ def _coerce_interval_list( value, keys, args, back_comp_unit_factor=1 ): | |||
| 'manual completion' : vdr( vtype='boolean', default=False ), | |||
| 'extra log files' : vdr( vtype='string_list', default=[] ), | |||
| 'enable resurrection' : vdr( vtype='boolean', default=False ), | |||
| 'work sub-directory' : vdr( vtype='string', default='$CYLC_TASK_ID' ), | |||
| 'work sub-directory' : vdr( vtype='string', default='$CYLC_TASK_CYCLE_TIME/$CYLC_TASK_NAME' ), | |||
|
For a task that does multiple retries, the gcylc log viewer "Choose Log File:" dropdown list now shows a bunch of identical entry names - we need to put the try numbers back in there. [update] actually it's worse than that - the logs retrieved are identical too (all the last one), not just their names. |
|
Aside from the above trivialities, looks great. |
Directory structure now looks like:
* `log/job/CYCLE/TASK/{01,02,...,NN}/{job,job.out,job.err,job.status}`
* `work/CYCLE/TASK/`
which should make it easier to archive the log for the whole cycle or to
housekeep all the work files of a given cycle.
`log/job/CYCLE/TASK/NN` is a symbolic link that points to the job log
directory of the latest submit for `TASK.CYCLE`.
It was displaying only the base names of the files, which would not be very meaningful when we have multiple submits. Rename CYCLE_TIME to CYCLE_POINT in `cylc.cfgspec.suite`.
|
Fixed the variable name and the combo viewer. N.B. I have re-based the branch against latest master. |
|
@matthewrmshin - is this ready to go after a final review on Monday? (you just said "take an initial look" above). |
|
Yes, provided that we are all happy with this backward incompatible change. |
|
[note - previous suggestion to drop the "job" filename prefix deleted - even though it is redundant, it clearly groups the cylc-generated files and distinguishes them from user-generated log files] |
I think we just need a one-off suite run directory conversion tool. Here's my attempt (done on top of your branch): dfd4055 |
|
(About the |
|
(I have merged in your one-off upgrade script.) |
Restructure suite run time log/job/ and work/
Directory structure now looks like:
log/job/CYCLE/TASK/{01,02,...,NN}/{job,job.out,job.err,job.status}work/CYCLE/TASK/which should make it easier to archive the log for the whole cycle or to
housekeep all the work files of a given cycle.
log/job/CYCLE/TASK/NNis a symbolic link that points to the job logdirectory of the latest submit for
TASK.CYCLE.Close #991.