upgrade decco to k8s v1.16.x APIs

[bcle]

The following resources have moved to newer API sets:

• Deployments
• Ingresses
• CRD (reverted because v1 requires validation schema, while v1beta1 is still supported until 1.19)

See https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.16.md#deprecations-and-removals for details

[bcle]

Testing on staging cluster reveals that CRD registration is failing and needs more work:

time="2020-05-19T22:33:02Z" level=error msg="resync failed: fail to create CRD: CustomResourceDefinition.apiextensions.k8s.io \"spaces.decco.platform9.com\" is invalid: spec.versions[0].schema.openAPIV3Schema: Required value: schemas are required" watchname=spaces-in-all-namespaces
time="2020-05-19T22:33:02Z" level=info msg="retry in 30s..." watchname=spaces-in-all-namespaces

[erwinvaneyk]

Wondering, don't the manifests (and manifest templates) also need to updated?

[bcle]

Wondering, don't the manifests (and manifest templates) also need to updated?

The only manifest that I see that would need updating is decco-apps-crd.yaml (the reason it exists as opposed to being created by the operator is a little obscure, it's a chicken-and-egg problem that I can explain if you like). But since I reverted the CRD changes for now (due to the need to supply a validation schema, which seems like it'll require some non-trivial work), there's no immediate need to do this now. Did you see any others that need updating?

[erwinvaneyk]

Ah ok, I thought we were also using manifests/templates/decco-deployment.yaml.tmpl, which would need updating to apps/v1 too.

Related to the validation schemas, it is something that kubebuilder generates for us in the Decco V2 work. So, if we merge #13, I can prepare the next PR that adds the validation schema to the CRDs.