Sawtooth memory usage of ingester

[bboreham]

We had an ingester restart abruptly, and its memory usage looks like this:

We have -max-chunk-age=9h and it's repeating on a 9h cycle. I think this is mostly the bigchunks growing over time then being flushed (and the flushed one is retained for 10 minutes or so).

I have an idea: for each series we could pick a slot within the 9h cycle, then always flush that series at that point. Now all flushing is evened out, at the cost of some short chunks right after a cold start.