Handle suite out and err files like state dumps?

[hjoliver]

ref #828 (comment)

[hjoliver]

[meeting] @oliver-sanders to investigate

[oliver-sanders]

A couple of quick questions to gauge opinion on. @cylc/core

1. Should we include the log file in with this, it would be nice to have all error files written out in the same way?
2. This could be achieved nicely if out and err were written through pythons logging module (would make Suite log, out, err rotation should be synchronized #1446 nice and simple and rationalize use of suite stdout, stderr, and the log #386 easier), how do we feel about this?

[hjoliver]

@oliver-sanders - are you suggesting using a single log file for all, or connecting stdout and stderr to different files handled by the logging module? (in the latter case, can the roll-over of multiple logs be synchronised?). I'm open to any suggestions as the current situation isn't very clean. However, @matthewrmshin seems to strongly dislike the Python logging module??

[oliver-sanders]

@hjoliver I was suggesting writing stdout and stderr messages through the logging module to their respective files (not merging them into one). If it was done like this synchronisation should be relatively straight forward to implement by extending the functionality of the logging module.

[hjoliver]

OK, sounds fine to me if others agree.

[matthewrmshin]

The logging module is appropriate for logging suite activity.

(I do have a dislike of it, because it is too flexible for the things we don't normally need, but incredibly rigid for the things we want to be flexible.)