Error when patching using patch_namespaced_custom_object()

[eric4321]

What happened (please include outputs or screenshots):

Hi, I am unable to patch a custom resource using patch_namespaced_custom_object(). I receive the error:

  File "/home/user/.local/lib/python3.8/site-packages/kubernetes/client/api/custom_objects_api.py", line 2031, in patch_namespaced_custom_object
    (data) = self.patch_namespaced_custom_object_with_http_info(group, version, namespace, plural, name, body, **kwargs)  # noqa: E501
  File "/home/user/.local/lib/python3.8/site-packages/kubernetes/client/api/custom_objects_api.py", line 2131, in patch_namespaced_custom_object_with_http_info
    return self.api_client.call_api(
  File "/home/user/.local/lib/python3.8/site-packages/kubernetes/client/api_client.py", line 340, in call_api
    return self.__call_api(resource_path, method,
  File "/home/user/.local/lib/python3.8/site-packages/kubernetes/client/api_client.py", line 172, in __call_api
    response_data = self.request(
  File "/home/user/.local/lib/python3.8/site-packages/kubernetes/client/api_client.py", line 398, in request
    return self.rest_client.PATCH(url,
  File "/home/user/.local/lib/python3.8/site-packages/kubernetes/client/rest.py", line 292, in PATCH
    return self.request("PATCH", url,
  File "/home/user/.local/lib/python3.8/site-packages/kubernetes/client/rest.py", line 231, in request
    raise ApiException(http_resp=r)
kubernetes.client.rest.ApiException: (500)
Reason: Internal Server Error
HTTP response headers: HTTPHeaderDict({'Audit-Id': 'fce1f9ff-9cfe-44c5-b0a7-d959c796a706', 'Content-Type': 'application/json', 'Date': 'Tue, 28 Jul 2020 14:23:37 GMT', 'Content-Length': '300'})
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"couldn't get version/kind; json parse error: json: cannot unmarshal array into Go value of type struct { APIVersion string \"json:\\\"apiVersion,omitempty\\\"\"; Kind string \"json:\\\"kind,omitempty\\\"\" }","code":500}

What you expected to happen:

For the patch to be applied successfully.

How to reproduce it (as minimally and precisely as possible):

Create a basic CRD:

$ cat <<EOF | kubectl apply -f -
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: tests.example.com
spec:
  group: example.com
  names:
    kind: Test
    listKind: TestList
    plural: tests
    shortNames:
    - tst
    singular: test
  scope: Namespaced
  validation:
    openAPIV3Schema:
      properties:
        spec:
          properties:
            exampleField:
              type: string
  version: v1
  versions:
  - name: v1
    served: true
    storage: true
EOF

Create a resource to test against:

$ cat <<EOF | kubectl apply -f -
apiVersion: "example.com/v1"
kind: Test
metadata:
  name: test-resource
  namespace: default
spec:
  exampleField: "originalvalue"
EOF

Run the following Python to try and change exampleField:

from kubernetes import client, config
import json

config.load_kube_config()
api = client.CustomObjectsApi()
body = [{"op": "replace", "path": "/spec/exampleField", "value": "updatefield"}]
response = api.patch_namespaced_custom_object(group = "example.com", version = "v1", namespace = "default", plural = "tests", name = "test-resource", body = body)

Anything else we need to know?:
The JSON patch works fine if applied using kubectl:

$ kubectl patch tests.example.com test-resource --type json -p "[{"op": "replace", "path": "/spec/exampleField", "value": "updatedfield"}]"
test.example.com/test-resource patched
$
$ kubectl get test test-resource -o yaml
apiVersion: example.com/v1
kind: Test
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"example.com/v1","kind":"Test","metadata":{"annotations":{},"name":"test-resource","namespace":"default"},"spec":{"exampleField":"originalvalue"}}
  creationTimestamp: "2020-07-28T15:03:34Z"
  generation: 2
  name: test-resource
  namespace: default
  resourceVersion: "71367992"
  selfLink: /apis/example.com/v1/namespaces/ddl/tests/test-resource
  uid: 821d86e2-d0e3-11ea-9049-4e9b43ee4c3e
spec:
  exampleField: updatedfield

Environment:

• Kubernetes version (kubectl version):
  • Client: 1.15.5
  • Server: 1.13.0 (AKS)
• OS (e.g., MacOS 10.13.6): Ubuntu 20.04.1 LTS
• Python version (python --version): Python 3.8.2
• Python client version (pip list | grep kubernetes): kubernetes 11.0.0

[roycaihw]

/assign

[roycaihw]

cc @yliaog

[yliaog]

In the following CRD spec, the kind: is 'Test' with uppercase 'T', could you trying switching it to lowercase, and see if you still see the same errors?

====================
spec:
group: example.com
names:
kind: Test
listKind: TestList
plural: tests
shortNames:
- tst
singular: test

[eric4321]

Hi @yliaog ,

I have followed your advice and tried with a lowercase 'test' in in 'kind' but it is failing with the same issue.

[ethuleau]

I think only merge patch is supported by the library [0].
But it seems json patch was added a year ago and introduced in 10.0.0 [1]. Probably reverted?

[0] https://github.com/kubernetes-client/python/blob/b5603d8ee2/kubernetes/client/api/custom_objects_api.py#L2408
[1] kubernetes-client/gen#119

[eric4321]

Thanks @doude,

I can see from the gen PR that "application/json-patch+json" was added but I can't see that a relevant change was made in this repository (mainly select_header_content_type() being able to add the new Content-Type header to the request.

Closing this as it is not an error, it is just unsupported.

[NikPaushkin]

If someone needs a solution, you could make this monkey patch of Kubernetes client.

class ApiClientForJsonPatch(client.ApiClient):
    def call_api(self, resource_path, method,
                 path_params=None, query_params=None, header_params=None,
                 body=None, post_params=None, files=None,
                 response_type=None, auth_settings=None, async_req=None,
                 _return_http_data_only=None, collection_formats=None,
                 _preload_content=True, _request_timeout=None):
        header_params['Content-Type'] = self.select_header_content_type(['application/json-patch+json'])
        return super().call_api(resource_path, method, path_params, query_params, header_params, body,
                                post_params, files, response_type, auth_settings, async_req, _return_http_data_only,
                                collection_formats, _preload_content, _request_timeout)

Then use ApiClientForJsonPatch class instead of native client.ApiClient for your jsonPatch requests.

[EvanKuhn-Affirm]

In case anyone else is still running into this...

I just dealt with this same problem. As a workaround, I had to call:

api.api_client.set_default_header('Content-Type', 'application/json-patch+json')  # Case-sensitive!

before api.patch_namespaced_custom_object.

This, along with passing the body as an object (I was originally passing it as a string) got things working.

This was with kubernetes==24.2.0