Support imagePullSecrets in ServingRuntime

[lizzzcai]

Is your feature request related to a problem? If so, please describe.

Most of our use cases are using custom servingRuntime which are built by the users. In this case, the servingRuntime image mostly will be stored in the user's private image registry.
However, the current ServingRuntime CRD doesn't have the imagePullSecrets, so we are not able to provide it to the user.

Describe your proposed solution

support imagePullSecrets under ServingRuntime.spec

Describe alternatives you have considered

Similiar to InferenceService CRD, ServingRuntime should be able to support most of the Pod Spec.

Additional context

[njhill]

@lizzzcai you can provide imagePullSecrets via the service account used for the deployment pods, which can be set via the serviceAccountName configuration parameter (see here). Is that sufficient or you need to be able to set different secrets per serving runtime?

[lizzzcai]

@lizzzcai you can provide imagePullSecrets via the service account used for the deployment pods, which can be set via the serviceAccountName configuration parameter (see here). Is that sufficient or you need to be able to set different secrets per serving runtime?

Hi @njhill , I tested this method but found that it is a system-wide configuration on the modelmesh-serving namespace. Hoever in our case, we are using user-namespace and each namespace will have multiple imagePullSecrets per namespace. So either this model-serving-config can be applied and overwrite in user namespace or enable the full PodSpec in servingruntime will be helpful. The current workaround for us is to use webhook to inject it, but it will be good to have a long term solution from kserve side.

[lizzzcai]

Hi @njhill , I have created a PR for this. I think it makes sense to add it as it is very common to use images from private repositories, and it is similar to volumes. What do you think?