Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: Skip InferenceService patching for KServe RawDeployment [RHOAIENG-10500] #262

Merged
merged 1 commit into from
Jul 29, 2024
Merged

fix: Skip InferenceService patching for KServe RawDeployment [RHOAIENG-10500] #262

merged 1 commit into from
Jul 29, 2024

Conversation

ruivieira
Copy link
Member

See RHOAIENG-10500.

Disable InferenceService configuration when its in RawDeployment mode.

@ruivieira ruivieira added the kind/enhancement New feature or request label Jul 27, 2024
@ruivieira ruivieira self-assigned this Jul 27, 2024
@ruivieira
Copy link
Member Author

/retest

@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 28, 2024 •
edited
Loading

PR image build and manifest generation completed successfully!

📦 PR image: quay.io/trustyai/trustyai-service-operator-ci:b341d58c3b7aba07bee6dce4ab958d0873f1ae6a

🗂️ CI manifests

@openshift-ci openshift-ci bot added the lgtm label Jul 29, 2024
@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Jul 29, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: RobGeada

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@RobGeada
Copy link
Contributor

/test trustyai-service-operator-e2e

1 similar comment
@RobGeada
Copy link
Contributor

/test trustyai-service-operator-e2e

@RobGeada
Copy link
Contributor

/test all

@ruivieira ruivieira merged commit ec4462e into trustyai-explainability:main Jul 29, 2024
6 of 9 checks passed
ruivieira added a commit that referenced this pull request Aug 5, 2024
@yhwang @ruivieira @RobGeada
sync: sync dev/lm-eval with main branch (#271)
427d102 
* feat: Initial database support (#246)

* Initial database support

- Add status checking
- Add better storage flags
- Add spec.storage.format validation
- Add DDL
-Add HIBERNATE format to DB (test)
- Update service image
- Revert identifier to DATABASE
- Update CR options (remove mandatory data)

* Remove default DDL generation env var

* Update service image to latest tag

* Add migration awareness

* Add updating pods for migration

* Change JDBC url from mysql to mariadb

* Fix TLS mount

* Revert images

* Remove redundant logic

* Fix comments

* feat: Add TLS certificate mount on ModelMesh (#255)

* feat: Add TLS certificate mount on ModelMesh

* Revert from http to https until kserve/modelmesh#147 is merged

* Pin oc version, ubi version (#263)

* Restore checkout of trustyai-exp (#265)

* Add operator installation robustness (#266)

* fix: Skip InferenceService patching for KServe RawDeployment (#262)

* feat: ConfigMap key to disable KServe Serverless configuration (#267)

* feat: Add support for custom certificates in database connection (#259)

* Add TLS endpoint for ModelMesh payload processors. (#268)

Keep non-TLS endpoint for KServe Serverless (disabled by default)

---------

Signed-off-by: Yihong Wang <yh.wang@ibm.com>
Co-authored-by: Rui Vieira <ruidevieira@googlemail.com>
Co-authored-by: Rob Geada <rob@geada.net>
ruivieira added a commit that referenced this pull request Aug 23, 2024
@yhwang @ruivieira @RobGeada
Weekly sync up of dev/lm-eval branch (#278)
342d1e2 
* feat: Initial database support (#246)

* Initial database support

- Add status checking
- Add better storage flags
- Add spec.storage.format validation
- Add DDL
-Add HIBERNATE format to DB (test)
- Update service image
- Revert identifier to DATABASE
- Update CR options (remove mandatory data)

* Remove default DDL generation env var

* Update service image to latest tag

* Add migration awareness

* Add updating pods for migration

* Change JDBC url from mysql to mariadb

* Fix TLS mount

* Revert images

* Remove redundant logic

* Fix comments

* feat: Add TLS certificate mount on ModelMesh (#255)

* feat: Add TLS certificate mount on ModelMesh

* Revert from http to https until kserve/modelmesh#147 is merged

* Pin oc version, ubi version (#263)

* Restore checkout of trustyai-exp (#265)

* Add operator installation robustness (#266)

* fix: Skip InferenceService patching for KServe RawDeployment (#262)

* feat: ConfigMap key to disable KServe Serverless configuration (#267)

* feat: Add support for custom certificates in database connection (#259)

* Add TLS endpoint for ModelMesh payload processors. (#268)

Keep non-TLS endpoint for KServe Serverless (disabled by default)

* fix: Correct maxSurge and maxUnavailable (#275)

* feat: Add support for custom DB names (#257)

* feat: Add support for custom DB names

* fix: Correct custom DB name

---------

Signed-off-by: Yihong Wang <yh.wang@ibm.com>
Co-authored-by: Rui Vieira <ruidevieira@googlemail.com>
Co-authored-by: Rob Geada <rob@geada.net>
ruivieira added a commit that referenced this pull request Oct 22, 2024
@ruivieira @yhwang
@RobGeada @tedhtchang @adolfo-ab
Merge LM-Eval dev branch (#337)
f70af36 
* Add lm-eval-service controller (#258)

* feat: Initial database support (#246)

* Initial database support

- Add status checking
- Add better storage flags
- Add spec.storage.format validation
- Add DDL
-Add HIBERNATE format to DB (test)
- Update service image
- Revert identifier to DATABASE
- Update CR options (remove mandatory data)

* Remove default DDL generation env var

* Update service image to latest tag

* Add migration awareness

* Add updating pods for migration

* Change JDBC url from mysql to mariadb

* Fix TLS mount

* Revert images

* Remove redundant logic

* Fix comments

* feat: Add TLS certificate mount on ModelMesh (#255)

* feat: Add TLS certificate mount on ModelMesh

* Revert from http to https until kserve/modelmesh#147 is merged

* Add lm-eval-service controller

refactor the existing TrustyAIService controller and
add LMEvalService controller

Signed-off-by: Yihong Wang <yh.wang@ibm.com>

---------

Signed-off-by: Yihong Wang <yh.wang@ibm.com>
Co-authored-by: Rui Vieira <ruidevieira@googlemail.com>

* fix: Fix typo in operator's arguments (#261)

Operator's arguments changed from `--eanble-services` to `--enable-services`.
trustyai.opendatahub.io_lmevaljobs.yaml and zz_generated.deepcopy.go regenerated.

* feat: Add LMES driver build to GHA (#272)

* sync: sync dev/lm-eval with main branch (#271)

* feat: Initial database support (#246)

* Initial database support

- Add status checking
- Add better storage flags
- Add spec.storage.format validation
- Add DDL
-Add HIBERNATE format to DB (test)
- Update service image
- Revert identifier to DATABASE
- Update CR options (remove mandatory data)

* Remove default DDL generation env var

* Update service image to latest tag

* Add migration awareness

* Add updating pods for migration

* Change JDBC url from mysql to mariadb

* Fix TLS mount

* Revert images

* Remove redundant logic

* Fix comments

* feat: Add TLS certificate mount on ModelMesh (#255)

* feat: Add TLS certificate mount on ModelMesh

* Revert from http to https until kserve/modelmesh#147 is merged

* Pin oc version, ubi version (#263)

* Restore checkout of trustyai-exp (#265)

* Add operator installation robustness (#266)

* fix: Skip InferenceService patching for KServe RawDeployment (#262)

* feat: ConfigMap key to disable KServe Serverless configuration (#267)

* feat: Add support for custom certificates in database connection (#259)

* Add TLS endpoint for ModelMesh payload processors. (#268)

Keep non-TLS endpoint for KServe Serverless (disabled by default)

---------

Signed-off-by: Yihong Wang <yh.wang@ibm.com>
Co-authored-by: Rui Vieira <ruidevieira@googlemail.com>
Co-authored-by: Rob Geada <rob@geada.net>

* Weekly sync up of dev/lm-eval branch (#278)

* feat: Initial database support (#246)

* Initial database support

- Add status checking
- Add better storage flags
- Add spec.storage.format validation
- Add DDL
-Add HIBERNATE format to DB (test)
- Update service image
- Revert identifier to DATABASE
- Update CR options (remove mandatory data)

* Remove default DDL generation env var

* Update service image to latest tag

* Add migration awareness

* Add updating pods for migration

* Change JDBC url from mysql to mariadb

* Fix TLS mount

* Revert images

* Remove redundant logic

* Fix comments

* feat: Add TLS certificate mount on ModelMesh (#255)

* feat: Add TLS certificate mount on ModelMesh

* Revert from http to https until kserve/modelmesh#147 is merged

* Pin oc version, ubi version (#263)

* Restore checkout of trustyai-exp (#265)

* Add operator installation robustness (#266)

* fix: Skip InferenceService patching for KServe RawDeployment (#262)

* feat: ConfigMap key to disable KServe Serverless configuration (#267)

* feat: Add support for custom certificates in database connection (#259)

* Add TLS endpoint for ModelMesh payload processors. (#268)

Keep non-TLS endpoint for KServe Serverless (disabled by default)

* fix: Correct maxSurge and maxUnavailable (#275)

* feat: Add support for custom DB names (#257)

* feat: Add support for custom DB names

* fix: Correct custom DB name

---------

Signed-off-by: Yihong Wang <yh.wang@ibm.com>
Co-authored-by: Rui Vieira <ruidevieira@googlemail.com>
Co-authored-by: Rob Geada <rob@geada.net>

* Driver updates job's status periodically (#280)

The driver periodically update the LMEvalJob.Status.Message field
with the outputs from the lm-eval. The message pattern the driver
captures is like `Running text generation:  81%|`. Then users
can use this information to check the progress of the job.

Signed-off-by: Yihong Wang <yh.wang@ibm.com>

* Add Dockerfile for LMES job image (#276)

Add Dockerfile for LMES job image and the needed files

Signed-off-by: Yihong Wang <yh.wang@ibm.com>

* feat: Add overlays (#283)

* feat: Add overlays

* Remove redundant lmes-tas overlay. Change job image name.

* Add job image build (#284)

* Change job image use midstream lm-evaluation-harness (#285)

* feat: support batch size (#290)

Add batch size support in the LMEvalJob which
leverages the `--batch_size` in the `lm-evaluation-harness`.
This only affects the local models. The `--bath_size` doesn't
work for remote inference APIs.

Signed-off-by: Yihong Wang <yh.wang@ibm.com>

* Add the `openai` package into the lmes job image (#292)

update the LMES job's Dockerfile to include the
`openai` package.

Signed-off-by: Yihong Wang <yh.wang@ibm.com>

* fix: fix dependency error in the job image (#296)

Split up the unitxt and openai dependencies to
avoid the conflict.

Signed-off-by: Yihong Wang <yh.wang@ibm.com>

* feat: add device detection in lmes driver (#298)

Added a new feature in LMES driver to detect the available
devices by using the PyTorch API. This feature can be disabled
by passing the `--detect-device false` option.

Signed-off-by: Yihong Wang <yh.wang@ibm.com>

* feat: support unitxt recipes (#301)

Add new fields in the CRD to support unitxt recipes and
leverage the driver to create corresponding yaml files
of the unitxt recipes.

Signed-off-by: Yihong Wang <yh.wang@ibm.com>

* feat: support custom dataset (#309)

Updated the CRD data struct to allow users to specify a custom Unitxt card in
JSON format. The custom Unitxt card is equivalent to a custom dataset
definition. Also restructured and updated the CRD to support Volumes,
VolumeMounts, Env, Resources, Labels, and Annotations.

Signed-off-by: Yihong Wang <yh.wang@ibm.com>

* feat: new pulling mechanism for job statuses (#314)

Update the driver to keep running even the user program
finishes. The driver provides two APIs:
- GetStatus(): retrieve job status
- Shutdown(): properly tear down the driver

In the controller side, it uses `pod/exec` resource
to run the driver command to invoke the driver APIs
to retrieve the job status and shutdown the driver
when job is done.

Signed-off-by: Yihong Wang <yh.wang@ibm.com>

* Move operator's cmd/operator/main.go to cmd/main.go to keep operator-sdk compatibility (#295)

* Remove hardcoded job's user ID (#322)

* Fix mkdir command in Job dockerfile (#330)

* Refactor some lmesreconcile methods (#323)

* Refactor lmes reconcile  optoins

Signed-off-by: ted chang <htchang@us.ibm.com>

* Update controllers/lmes/lmevaljob_controller.go

Co-authored-by: Yihong Wang <yh.wang@ibm.com>

* Update controllers/lmes/lmevaljob_controller.go

Co-authored-by: Yihong Wang <yh.wang@ibm.com>
Signed-off-by: ted chang <htchang@us.ibm.com>

---------

Signed-off-by: ted chang <htchang@us.ibm.com>
Co-authored-by: Yihong Wang <yh.wang@ibm.com>

* tidy: clean up lmes-job image (#333)

remove BAM related packages and patch.

Signed-off-by: Yihong Wang <yh.wang@ibm.com>

* Enable job suspend for Kueue (#317)

* Refactor lmes reconcile  optoins

Signed-off-by: ted chang <htchang@us.ibm.com>

* Update controllers/lmes/lmevaljob_controller.go

Co-authored-by: Yihong Wang <yh.wang@ibm.com>

* Update controllers/lmes/lmevaljob_controller.go

Co-authored-by: Yihong Wang <yh.wang@ibm.com>
Signed-off-by: ted chang <htchang@us.ibm.com>

* Enable job suspend for Kueue

Signed-off-by: ted chang <htchang@us.ibm.com>

---------

Signed-off-by: ted chang <htchang@us.ibm.com>
Co-authored-by: Yihong Wang <yh.wang@ibm.com>

* Add overlay placeholders for main merge (#334)

* sync: sync up dev/lm-eval branch with main branch (#336)

* [CI] Run tests from trustyai-tests (#279)

* Change Dockerfile to clone trustyai-tests

* Add PYTEST_MARKERS env and remove TESTS_REGEX

* RHOAIENG-12274: Update operator's overlays (#287)

* Update operator's overlays

* Update kustomization.yaml

* Add devflag printout to GH Action comment (#289)

* Add timeout loop to DSC install (#305)

* RHOAIENG-13625: Add DBAvailable status to CR (#304)

* Add DBAvailable status to CR

* Remove probes

* Add KServe destination rule for Inference Services in the ServiceMesh (#315)

* Add DestinationRule creation for KServe serverless

* Add permissions for destination rules

* Add role for destination rules

* Add missing role for creating destination rules

* Fix spacing in DestinationRule template

* Add check if DestinationRule CRD is present before creating it (#316)

* Add check for DestinationRule CRD

* Add API extensions to operator's scheme

* Add permission for CRD resource

* Fix operator metrics service target port (#320)

* Add readiness probes (#312)

* Enable KServe serverless in the rhoai overlay (#321)

* Update overlay images (#331)

* Add correct CA cert to JDBC (#324)

* Add correct CA cert to JDBC

* Add require SSL

* Support for VirtualServices for InferenceLogger traffic (#332)

* Generate KServe Inference Logger in conformance with DestinationRule and VirtualService

* Add VirtualService creation for models in the mesh

* Add permissions for VirtualServices

* Update manifests for VirtualServices

* Fix VirtualServiceName variable

* fix yaml linter after the sync

Signed-off-by: Yihong Wang <yh.wang@ibm.com>

* tidy the go.mod and go.sum as well

Signed-off-by: Yihong Wang <yh.wang@ibm.com>

---------

Signed-off-by: Yihong Wang <yh.wang@ibm.com>
Co-authored-by: Adolfo Aguirrezabal <aaguirre@redhat.com>
Co-authored-by: Rui Vieira <ruidevieira@googlemail.com>
Co-authored-by: Rob Geada <rob@geada.net>
Co-authored-by: Rui Vieira <rcardoso@redhat.com>

---------

Signed-off-by: Yihong Wang <yh.wang@ibm.com>
Signed-off-by: ted chang <htchang@us.ibm.com>
Co-authored-by: Yihong Wang <yh.wang@ibm.com>
Co-authored-by: Rob Geada <rob@geada.net>
Co-authored-by: ted chang <htchang@us.ibm.com>
Co-authored-by: Adolfo Aguirrezabal <aaguirre@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@RobGeada RobGeada RobGeada approved these changes

@danielezonca danielezonca Awaiting requested review from danielezonca

@christinaexyou christinaexyou Awaiting requested review from christinaexyou

Assignees

@ruivieira ruivieira

@RobGeada RobGeada

Labels
kind/enhancement New feature or request lgtm ok-to-test
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ruivieira @RobGeada