[v0.14] Cherry-pick Inference Graph stabilizations #427
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
spolti
reviewed
Nov 4, 2024
3f572cc to
ae39d23
Compare
spolti
approved these changes
Nov 28, 2024
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: israel-hdez, spolti The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
This changes kserve-controller to mount the OpenShift Service CA bundle into kserve-router container and a configures it to trust the bundle. This affects InferenceGraph deployed in Serverless mode. With these changes, InferenceGraphs will work correctly when deployed without an Istio sidecar. These changes are needed because in ODH the InferenceServices are secured with TLS. The internal endpoints (which are the ones InferenceGraph uses) are using OpenShift service serving certificates. Related to: https://issues.redhat.com/browse/RHOAIENG-13448 Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>
In topologies using Istio mesh, applications can use plain-text HTTP to send traffic to other mesh-member workloads. Handling of TLS is delegated to Istio. Thus, when KServe workloads have the Istio sidecar (e.g. by using auto-injection), the Inference Graph router should send its traffic without TLS even if the schema of the service URL is specified as HTTPS. Istio would originate TLS when needed, and the double TLS is prevented (see https://istio.io/latest/docs/ops/common-problems/network-issues/#double-tls). These changes implement a detection of the Istio sidecar by querying a well known port that the sidecar is using. If the sidecar is found, inference requests are sent using plain-text HTTP. Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>
ae39d23 to
1ce708d
Compare
66b6202
into
opendatahub-io:release-v0.14
19 of 20 checks passed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Cherry-pick the following PRs:
Which issue(s) this PR fixes (optional, in
fixes #<issue number>(, fixes #<issue_number>, ...)format, will close the issue(s) when PR gets merged):Fixes https://issues.redhat.com/browse/RHOAIENG-13448
Type of changes
Please delete options that are not relevant.