Parallelize local memory arbitration #9649
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for meta-velox canceled.
|
|
@xiaoxmeng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
tanjialiang
reviewed
Apr 30, 2024
tanjialiang
reviewed
Apr 30, 2024
tanjialiang
reviewed
Apr 30, 2024
tanjialiang
reviewed
Apr 30, 2024
tanjialiang
reviewed
Apr 30, 2024
tanjialiang
reviewed
Apr 30, 2024
tanjialiang
reviewed
Apr 30, 2024
tanjialiang
reviewed
Apr 30, 2024
| }; | ||
|
|
||
| // Invoked to check if the memory growth will exceed the memory pool's max | ||
| // capacity limit or the arbitrator's node capacity limit. | ||
| bool checkCapacityGrowth(const MemoryPool& pool, uint64_t targetBytes) const; | ||
| bool checkCapacityGrowth(ArbitrationOperation* op) const; |
There was a problem hiding this comment.
Instead of passing around the same object everywhere, shall we just pass this operation object to runLocalArbitration() and keep track of an instance of operation of currently running global one? It seems unnecessary to pass this stateful item around for global aribtration.
There was a problem hiding this comment.
We can set the current running arbitration operation as a member of shared arbitrator object but there a quite a few utilities shared between local and global arbitration operations.
There was a problem hiding this comment.
looks like the only shared one is checkFreeCapacity()?
tanjialiang
reviewed
Apr 30, 2024
tanjialiang
reviewed
Apr 30, 2024
|
@xiaoxmeng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
tanjialiang
reviewed
Apr 30, 2024
tanjialiang
reviewed
Apr 30, 2024
tanjialiang
reviewed
Apr 30, 2024
tanjialiang
reviewed
May 1, 2024
tanjialiang
reviewed
May 1, 2024
tanjialiang
reviewed
May 1, 2024
tanjialiang
reviewed
May 1, 2024
tanjialiang
reviewed
May 1, 2024
tanjialiang
reviewed
May 1, 2024
50e0ce7 to
ce4d9f0
Compare
|
@xiaoxmeng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@xiaoxmeng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@xiaoxmeng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@xiaoxmeng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
xiaoxmeng
added a commit
to xiaoxmeng/velox
that referenced
this pull request
May 4, 2024
Summary: Parallelize the local memory arbitration execution. The workload flow of memory arbitration process changes as follows: First wait in the arbitration queue to serialize the memory arbitration request processing from the same query pool. Adds ArbitrationQueue data structure for this which contains the wait promises of the arbitration requests from the same query pool. The arbitration queue is protected by arbitrator lock. Second calls runLocalArbitration to run local arbitration which acquires the reader lock of arbitration lock (which is added by this pr for local/global arbitration execution control). Then it ensures the request memory pool is within the capacity limit (this might trigger spill to reclaim the used memory from the request pool itself), and tries to allocate free capacity from the arbitrator or reclaim the free memory from the other queries which is protected by arbitrator lock. Third if runLocalArbitration can't reclaim sufficient memory, then proceeds with runGlobalArbitration which acquires the writer lock of arbitration lock. Then it reclaims the free capacity from the arbitrator or the other queries. And at last reclaims the used memory from the other queries by spilling. ArbitrationOperation is added to contains the state of arbitration request processing to simplify the code implementation Reviewed By: tanjialiang, oerling Differential Revision: D56685200 Pulled By: xiaoxmeng
|
This pull request was exported from Phabricator. Differential Revision: D56685200 |
d2f1781 to
cef833d
Compare
|
@xiaoxmeng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@xiaoxmeng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
xiaoxmeng
added a commit
to xiaoxmeng/velox
that referenced
this pull request
May 7, 2024
Summary: Parallelize the local memory arbitration execution. The workload flow of memory arbitration process changes as follows: First wait in the arbitration queue to serialize the memory arbitration request processing from the same query pool. Adds ArbitrationQueue data structure for this which contains the wait promises of the arbitration requests from the same query pool. The arbitration queue is protected by arbitrator lock. Second calls runLocalArbitration to run local arbitration which acquires the reader lock of arbitration lock (which is added by this pr for local/global arbitration execution control). Then it ensures the request memory pool is within the capacity limit (this might trigger spill to reclaim the used memory from the request pool itself), and tries to allocate free capacity from the arbitrator or reclaim the free memory from the other queries which is protected by arbitrator lock. Third if runLocalArbitration can't reclaim sufficient memory, then proceeds with runGlobalArbitration which acquires the writer lock of arbitration lock. Then it reclaims the free capacity from the arbitrator or the other queries. And at last reclaims the used memory from the other queries by spilling. ArbitrationOperation is added to contains the state of arbitration request processing to simplify the code implementation. This PR also adds an option to disable global arbitration and only do local arbitration which can always run in parallel. With this option, for 2hrs stress test (1000 query concurrency at coordinator) with Meta internal production workloads replay, the averaged query execution time has been reduced by ~30% and cpu time is kept the same. The memory arbitration wall time for query without triggering spilled is only 9 mins in total. Reviewed By: tanjialiang, oerling Differential Revision: D56685200 Pulled By: xiaoxmeng
|
This pull request was exported from Phabricator. Differential Revision: D56685200 |
Summary: Parallelize the local memory arbitration execution. The workload flow of memory arbitration process changes as follows: First wait in the arbitration queue to serialize the memory arbitration request processing from the same query pool. Adds ArbitrationQueue data structure for this which contains the wait promises of the arbitration requests from the same query pool. The arbitration queue is protected by arbitrator lock. Second calls runLocalArbitration to run local arbitration which acquires the reader lock of arbitration lock (which is added by this pr for local/global arbitration execution control). Then it ensures the request memory pool is within the capacity limit (this might trigger spill to reclaim the used memory from the request pool itself), and tries to allocate free capacity from the arbitrator or reclaim the free memory from the other queries which is protected by arbitrator lock. Third if runLocalArbitration can't reclaim sufficient memory, then proceeds with runGlobalArbitration which acquires the writer lock of arbitration lock. Then it reclaims the free capacity from the arbitrator or the other queries. And at last reclaims the used memory from the other queries by spilling. ArbitrationOperation is added to contains the state of arbitration request processing to simplify the code implementation. This PR also adds an option to disable global arbitration and only do local arbitration which can always run in parallel. With this option, for 2hrs stress test (1000 query concurrency at coordinator) with Meta internal production workloads replay, the averaged query execution time has been reduced by ~30% and cpu time is kept the same. The memory arbitration wall time for query without triggering spilled is only 9 mins in total. Reviewed By: tanjialiang, oerling Differential Revision: D56685200 Pulled By: xiaoxmeng
|
This pull request was exported from Phabricator. Differential Revision: D56685200 |
|
@xiaoxmeng merged this pull request in 5911129. |
|
Conbench analyzed the 1 benchmark run on commit There were no benchmark performance regressions. 🎉 The full Conbench report has more details. |
Joe-Abraham
pushed a commit
to Joe-Abraham/velox
that referenced
this pull request
May 8, 2024
Summary: Parallelize the local memory arbitration execution. The workload flow of memory arbitration process changes as follows: First wait in the arbitration queue to serialize the memory arbitration request processing from the same query pool. Adds ArbitrationQueue data structure for this which contains the wait promises of the arbitration requests from the same query pool. The arbitration queue is protected by arbitrator lock. Second calls runLocalArbitration to run local arbitration which acquires the reader lock of arbitration lock (which is added by this pr for local/global arbitration execution control). Then it ensures the request memory pool is within the capacity limit (this might trigger spill to reclaim the used memory from the request pool itself), and tries to allocate free capacity from the arbitrator or reclaim the free memory from the other queries which is protected by arbitrator lock. Third if runLocalArbitration can't reclaim sufficient memory, then proceeds with runGlobalArbitration which acquires the writer lock of arbitration lock. Then it reclaims the free capacity from the arbitrator or the other queries. And at last reclaims the used memory from the other queries by spilling. ArbitrationOperation is added to contains the state of arbitration request processing to simplify the code implementation. This PR also adds an option to disable global arbitration and only do local arbitration which can always run in parallel. With this option, for 2hrs stress test (1000 query concurrency at coordinator) with Meta internal production workloads replay, the averaged query execution time has been reduced by ~30% and cpu time is kept the same. The memory arbitration wall time for query without triggering spilled is only 9 mins in total. Pull Request resolved: facebookincubator#9649 Reviewed By: tanjialiang, oerling Differential Revision: D56685200 Pulled By: xiaoxmeng fbshipit-source-id: f8db71e4cae05f24f913464c37c5bc1e6528083b
Joe-Abraham
pushed a commit
to Joe-Abraham/velox
that referenced
this pull request
Jun 7, 2024
Summary: Parallelize the local memory arbitration execution. The workload flow of memory arbitration process changes as follows: First wait in the arbitration queue to serialize the memory arbitration request processing from the same query pool. Adds ArbitrationQueue data structure for this which contains the wait promises of the arbitration requests from the same query pool. The arbitration queue is protected by arbitrator lock. Second calls runLocalArbitration to run local arbitration which acquires the reader lock of arbitration lock (which is added by this pr for local/global arbitration execution control). Then it ensures the request memory pool is within the capacity limit (this might trigger spill to reclaim the used memory from the request pool itself), and tries to allocate free capacity from the arbitrator or reclaim the free memory from the other queries which is protected by arbitrator lock. Third if runLocalArbitration can't reclaim sufficient memory, then proceeds with runGlobalArbitration which acquires the writer lock of arbitration lock. Then it reclaims the free capacity from the arbitrator or the other queries. And at last reclaims the used memory from the other queries by spilling. ArbitrationOperation is added to contains the state of arbitration request processing to simplify the code implementation. This PR also adds an option to disable global arbitration and only do local arbitration which can always run in parallel. With this option, for 2hrs stress test (1000 query concurrency at coordinator) with Meta internal production workloads replay, the averaged query execution time has been reduced by ~30% and cpu time is kept the same. The memory arbitration wall time for query without triggering spilled is only 9 mins in total. Pull Request resolved: facebookincubator#9649 Reviewed By: tanjialiang, oerling Differential Revision: D56685200 Pulled By: xiaoxmeng fbshipit-source-id: f8db71e4cae05f24f913464c37c5bc1e6528083b
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Parallelize the local memory arbitration execution. The workload flow of memory arbitration process
changes as follows:
First wait in the arbitration queue to serialize the memory arbitration request processing from the same
query pool. Adds ArbitrationQueue data structure for this which contains the wait promises of the
arbitration requests from the same query pool. The arbitration queue is protected by arbitrator lock.
Second calls runLocalArbitration to run local arbitration which acquires the reader lock of arbitration
lock (which is added by this pr for local/global arbitration execution control). Then it ensures the request
memory pool is within the capacity limit (this might trigger spill to reclaim the used memory from the
request pool itself), and tries to allocate free capacity from the arbitrator or reclaim the free memory from
the other queries which is protected by arbitrator lock.
Third if runLocalArbitration can't reclaim sufficient memory, then proceeds with runGlobalArbitration which
acquires the writer lock of arbitration lock. Then it reclaims the free capacity from the arbitrator or the other
queries. And at last reclaims the used memory from the other queries by spilling.
ArbitrationOperation is added to contains the state of arbitration request processing to simplify the code
implementation.
This PR also adds an option to disable global arbitration and only do local arbitration which can always
run in parallel. With this option, for 2hrs stress test (1000 query concurrency at coordinator) with Meta
internal production workloads replay, the averaged query execution time has been reduced by ~30% and cpu
time is kept the same. The memory arbitration wall time for query without triggering spilled is only 9 mins in
total.