Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

reading from GCS parallelly is slow, the speed is about the same as reading serially. #49340

Closed
D3Hunter opened this issue Dec 11, 2023 · 0 comments · Fixed by #49341
Closed

reading from GCS parallelly is slow, the speed is about the same as reading serially. #49340

D3Hunter opened this issue Dec 11, 2023 · 0 comments · Fixed by #49341
Labels
type/enhancement The issue or PR belongs to an enhancement.

Comments

@D3Hunter
Copy link
Contributor

Enhancement

reading from GCS parallelly is slow, the speed is about the same as reading serially.
after investigate, we found that http2 will reuse connection to send request, so parallelly readers is actually using 1 connection, we can enhance by set disable keep alive=true

see googleapis/google-cloud-go#1514 too
@D3Hunter D3Hunter added the type/enhancement The issue or PR belongs to an enhancement. label Dec 11, 2023
@D3Hunter D3Hunter mentioned this issue Dec 11, 2023
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Dec 12, 2023
@D3Hunter
br/storage: disable keep alive for GCS on default to speed up request (…
8c8333a 
…#49341)

close #49340
@D3Hunter D3Hunter mentioned this issue Dec 12, 2023
Merged
13 tasks
ti-chi-bot bot pushed a commit that referenced this issue Dec 12, 2023
@D3Hunter
br/storage: add scope to support service_account type of credential f…
9322568 
…ile & remove default GCS httpclient (#49363)

ref #49340
@lance6716 lance6716 mentioned this issue Jan 15, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

br/storage: disable keep alive for GCS on default to speed up request D3Hunter/tidb
1 participant
@D3Hunter