Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Fix download bugs during download benchmark #134

Merged
merged 7 commits into from
Sep 6, 2024

Conversation

brosoul
Copy link
Collaborator

@brosoul brosoul commented Sep 6, 2024

Pull Request Description

  1. TOS set multi part size should be part_size parameter
  2. Add environment variable TOS_ENABLE_CRC to control whether do CRC64 during TOS download is performed or not
  3. Added dependencies in runtime.Dockerfile. ensure crcmod._crcfunext could be imported correctly
  4. Avoid warning log "Connection pool is full"

Benchmark

AI Runtime can provide download speeds close to official tools and support downloading files from multiple storage sources.
Here is the detail result of downloading the benchmark for AI Runtime

Download 10G file from AWS S3

Comparing the use of AI Runtime, AWS CLI (offical download cli), and s5cmd to download from AWS S3

  • When CRT (Common Runtime tool) is turned off, AI Runtime can reach speeds similar to S5cmd, both of which are better than AWS CLI
  • When CRT is turned on, AWS CLI could be better than AI Runtime
    image

Download 10G file from TOS

Comparing the use of AI Runtime, tosutil (offical download cli)

  • When CRC64 checksum (ensure file integrity) is turned off, AI Runtime performance close to the tosutil
  • When CRC64 checksum is turned on, AI Runtime is slightly slower than the tosutil
    image

Related Issues

Resolves: #105

Important: Before submitting, please complete the description above and review the checklist below.


Contribution Guidelines (Expand for Details)

We appreciate your contribution to aibrix! To ensure a smooth review process and maintain high code quality, please adhere to the following guidelines:

Pull Request Title Format

Your PR title should start with one of these prefixes to indicate the nature of the change:

  • [Bug]: Corrections to existing functionality
  • [CI]: Changes to build process or CI pipeline
  • [Docs]: Updates or additions to documentation
  • [API]: Modifications to aibrix's API or interface
  • [CLI]: Changes or additions to the Command Line Interface
  • [Misc]: For changes not covered above (use sparingly)

Note: For changes spanning multiple categories, use multiple prefixes in order of importance.

Submission Checklist

  • PR title includes appropriate prefix(es)
  • Changes are clearly explained in the PR description
  • New and existing tests pass successfully
  • Code adheres to project style and best practices
  • Documentation updated to reflect changes (if applicable)
  • Thorough testing completed, no regressions introduced

By submitting this PR, you confirm that you've read these guidelines and your changes align with the project's contribution standards.

@brosoul brosoul requested a review from Jeffwan September 6, 2024 10:04
@Jeffwan
Copy link
Collaborator

Jeffwan commented Sep 6, 2024

One minor comment from my side.

@Jeffwan Jeffwan merged commit 62d21a0 into main Sep 6, 2024
3 checks passed
@Jeffwan Jeffwan deleted the linhui/fix-downloader branch September 6, 2024 12:49
gangmuk pushed a commit that referenced this pull request Jan 25, 2025
* feat: add tos downloader options enable_crc

* fix: fix tos part_size

* style: add log for download

* fix: make tos support crc extension

* fix: max_pool_connections settings

* style

* fix: dockerfile best practice
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Benchmark for AI Runtime models downloading from different sources
2 participants