Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Package AI Runtime and make it deployable as a sidecar #89

Closed
Jeffwan opened this issue Aug 21, 2024 · 0 comments · Fixed by #118
Closed

Package AI Runtime and make it deployable as a sidecar #89

Jeffwan opened this issue Aug 21, 2024 · 0 comments · Fixed by #118
Assignees
@brosoul
Labels
area/runtime priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now.
Milestone
v0.1.0-rc.1

Comments

@Jeffwan
Copy link
Collaborator

Jeffwan commented Aug 21, 2024 •
edited
Loading

🚀 Feature Description and Motivation

We should provide

  • the Dockerfile to package the python codes as container images.
  • the deployment yaml and practice for the sidecar.
  • Consider sidecar enable or disable workflow

The done criteria is to deploy the sidecar with vLLM engine. Let's make sure the sidecar can take the role of model downloading.

Use Case

No response

Proposed Solution

No response
@Jeffwan Jeffwan added priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. area/runtime labels Aug 22, 2024
@Jeffwan Jeffwan modified the milestones: v0.1.0, v0.1.0-rc.1 Aug 29, 2024
@brosoul brosoul mentioned this issue Sep 2, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@brosoul brosoul

Labels
area/runtime priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now.
Projects
None yet
Milestone
v0.1.0-rc.1
Development

Successfully merging a pull request may close this issue.

[CI] Package AI Runtime vllm-project/aibrix
2 participants
@Jeffwan @brosoul