Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Benchmark for AI Runtime models downloading from different sources #105

Closed
brosoul opened this issue Aug 29, 2024 · 3 comments · Fixed by #134
Closed

Benchmark for AI Runtime models downloading from different sources #105

brosoul opened this issue Aug 29, 2024 · 3 comments · Fixed by #134
Assignees
@brosoul
Labels
area/runtime kind/feature Categorizes issue or PR as related to a new feature. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now.
Milestone
v0.1.0-rc.1

Comments

@brosoul
Copy link
Collaborator

brosoul commented Aug 29, 2024 •
edited
Loading

🚀 Feature Description and Motivation

It is part of #81 Benchmark should include

  • support different storage (HuggingFace, AWS S3, TOS)
  • compare AI Runtime download speed with CLI
    • aws-cli, aws-cli[crt], s5cmd download from S3
    • tosutil download from TOS

Use Case

No response

Proposed Solution

No response
@brosoul brosoul self-assigned this Aug 29, 2024
@Jeffwan
Copy link
Collaborator

Jeffwan commented Aug 29, 2024

Let's split into two tasks. 1. storage backend support 2. benchmark

@Jeffwan Jeffwan added this to the v0.1.0-rc.1 milestone Aug 29, 2024
@Jeffwan Jeffwan added kind/feature Categorizes issue or PR as related to a new feature. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. labels Aug 29, 2024
@Jeffwan
Copy link
Collaborator

Jeffwan commented Aug 29, 2024

I split a separate task #109 for development. Let's keep this one for benchmarking

@Jeffwan Jeffwan changed the title Benchmark for AI Runtime download models Benchmark for AI Runtime models downloading from different sources Aug 29, 2024
@brosoul
Copy link
Collaborator Author

brosoul commented Sep 4, 2024

I will provide the benchmark results around this week.

This was referenced Sep 4, 2024
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@brosoul brosoul

Labels
area/runtime kind/feature Categorizes issue or PR as related to a new feature. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now.
Projects
None yet
Milestone
v0.1.0-rc.1
Development

Successfully merging a pull request may close this issue.

[Bug] Fix download bugs during download benchmark vllm-project/aibrix
2 participants
@Jeffwan @brosoul