Add GUC 'gp_random_insert_segments' to control the segments used for random distributed table insertion #406
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
519d20c to
50efffc
Compare
my-ship-it
reviewed
Apr 11, 2024
gfphoenix78
reviewed
Apr 12, 2024
…random distributed table insertion Introduces the 'gp_random_insert_segments' GUC to reduce the generation of excessive fragmented files during the insertion of small amounts of data into clusters with a large number of segments (e.g., 1000 records into 100 segments). Fragmented data insertion can significantly degrade performance, especially when using append-optimized or cloud-based storage. By introducing the 'gp_random_insert_segments' GUC, users can limit the number of segments used for data insertion in randomly distributed tables, which can significantly reduce fragmented files.
50efffc to
690b060
Compare
Update
|
|
Please add test case? |
foreyes
added a commit
to foreyes/cloudberrydb
that referenced
this pull request
Jul 8, 2024
…random distributed table insertion (apache#406) Introduces the 'gp_random_insert_segments' GUC to reduce the generation of excessive fragmented files during the insertion of small amounts of data into clusters with a large number of segments (e.g., 1000 records into 100 segments). Fragmented data insertion can significantly degrade performance, especially when using append-optimized or cloud-based storage. By introducing the 'gp_random_insert_segments' GUC, users can limit the number of segments used for data insertion in randomly distributed tables, which can significantly reduce fragmented files.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Introduces the 'gp_random_insert_segments' GUC to reduce the generation of excessive fragmented files during the insertion of small amounts of data into clusters with a large number of segments (e.g., 1000 records into 100 segments).
Fragmented data insertion can significantly degrade performance, especially when using append-optimized or cloud-based storage. By introducing the 'gp_random_insert_segments' GUC, users can limit the number of segments used for data insertion in randomly distributed tables, which can significantly reduce fragmented files.