Improve internal row to columnar host memory by using a combined spillable buffer

[jbrennan333]

This improves the host memory oom handling for InternalRowToColumnarBatchIterator by using a single spillable host buffer for both the data and offsets, and then slicing it to pull out the individual host buffers whenever we need to operate on them.

This simplifies the code and makes the host memory handling work better. Previously we were allocating one host buffer and making it spillable, and then allocating another buffer and making it spillable, all inside a single withRetry block. By doing this, we were sometimes able to allocate the second only because we spilled the first, and we really need both or nothing. When we later tried to get both host buffers from the two separate spillable buffers, we more frequently hit OOM with no retries.

Prior to this change, I needed about 32GB host memory limit run q75 and q10 with ShuffleExchangeExec operator disabled on my desktop at scale 100 with 16 executor cores. With this change, I can run them with a 12GB host memory limit.

I've run unit tests and integration tests for this and tested q10 and q75 locally.
I am in the process of running the full nds benchmark with host memory restricted.

[jbrennan333]

build

[jbrennan333]

I have run power run at scale 100 on my desktop with ShuffleExchangeExec disabled (to force more row to columnar conversions) and with host memory limited to 24g. I am not seeing any ooms in InternalRowToColumnarBatchIterator, but I do see some in other places (ColumnarToRowIterator.loadNextBatch and AcceleratedColumnarToRowIterator.setupBatchAndClose). I don't see these failures if I don't disable ShuffleExchangeExec.

[jbrennan333]

I was able to run the full power-run at scale 100 with host memory limited to 8g with no ooms (this was without disabling ShuffleExchangeExec, so it doesn't test this change much - just following up on my previous comment.

[sameerz]

Was there any performance impact seen when running the NDS tests?

[jbrennan333]

Was there any performance impact seen when running the NDS tests?

This code is not normally exercised with nds, unless I disable the ShuffleExchangeExec operator. I did not do a performance comparison for this. I think it is unlikely that there would be any noticeable change except in restricted host memory situations, where this will do better because it reduces thrashing when tasks get only one of the buffers they need.