Incomplete observing log

[virajkaram]

At the end of real-time running of pipeline, the observing csv log has only one science frame and all calibration frames. Presumably, it gets overwritten every time pipeline is run

[robertdstein]

I was thinking about this today. It will take a bit of work to get it right. First step would be to move the CSV log step to "load raw image" block. That prevents the repeated/useless overwriting. The more complicated thing is working out how to make the log properly. My personal idea would be to let the monitor execute a separate "finish up" block after n hours, which would be in charge of making the log and other such tasks like calculating quality.

[robertdstein]

In principal we could just reprocess the entire night on the spot at that time, using the calibration images from that night. Right now the monitor is not smart enough to work out that when new calibration images come in, it should stop using the ones from yesterday.