Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: create the workspace service for custom models #745

Conversation

zawachte
Copy link
Contributor

Reason for Change:

This change updates the ensureService function in the workspace controller to create the workspace service for custom models instead of only doing it for preset models.

The logic today only goes into the service creation code if there is a preset, but it is only using the preset information to see if the model supports distributed inference. To add support for custom models, we can just default the distributed inference support to false and always create the service preset or not.

It seems fair to me to only support distributed inference on the preset models as that requires more setup. With this change custom models can at least have basic support without any manual Kubernetes resource creation.

Fixes #744.

Requirements

  • added unit tests and e2e tests (if applicable).

Issue Fixed:

Fixes #744

Notes for Reviewers:

@zhuangqh zhuangqh changed the title Create the workspace service for custom models fix: create the workspace service for custom models Dec 2, 2024
Copy link
Collaborator

@zhuangqh zhuangqh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution!

@Fei-Guo Fei-Guo merged commit 1b440d5 into kaito-project:main Dec 2, 2024
4 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Custom Model Integration Doesn't Create a Workspace Service
3 participants