Prototype 3: API Author Demo

[pweil-]

Demo Objective

Close over consumer details, evolve APIs

1. For ingress (shouldn’t ingress have a wildcard?)
2. For serving (reencrypt)
3. For talking to another service

Demo Steps

1. Create new workspace for our new cert-manager API backend
2. Run cert-manager (as Deployment if possible, locally is also fine because not focus of this epic) with a kubeconfig of a kcp controller workspace.
3. Export the cert-manager APIs (those needed to issue a serving cert)
4. In another workspace, import the cert-manager APIs from above
5. Show webapp getting a certificate within the workspace
6. Stetch goal: A compatible API schema change is deployed
7. Stetch goal: See the workspaces to see the new schema automatically

Action Items

• Scope the current demo as necessary to fit in prototype boundaries
• Create and link required tasks to realize demo
• apis: add APIExport/Binding/ResourceSchema types #583 @sttts
• automatically install CRDs into Universal workspaces
• Create APIBindingController: @ncdc Add APIBinding controller, replace inheritFrom with APIBindings #755
  • create a CRD from APIResourceSchema in shadow workspace
  • phase transitions of the APIBinding
  • follow-ups:
    • [ ] set the storage versions in the APIBinding's status @ncdc moved to Follow-up APIBindings: set the storage versions in the APIBinding's status #842
    • stop unbinding, only unbinding we want is deletion, see https://github.com/kcp-dev/kcp/pull/702/files#r830041070
    • replace CRDReady condition with Established, see WIP: Add APIBinding controller, replace inheritFrom with APIBindings #702 (comment)
    • [ ] have more speaking names of CRDs in shadow namespace, see WIP: Add APIBinding controller, replace inheritFrom with APIBindings #702 (comment) moved Follow-up APIBindings: have more speaking names of CRDs in shadow namespace #843
    • consolodate system CRD installation in couple of post-start-hooks and by system CRD provider, see kcp-install-api-resource-controller
    • [ ] add admission to only allow apis.kcp.dev/bound-crd annotation in system:bound-crds workspace https://github.com/kcp-dev/kubernetes/pull/48/files#r827736066 moved to Follow-up APIBindings: Add admission to only allow apis.kcp.dev/bound-crd annotation in system:bound-crds workspace #844
    • [ ] stop CRDs and APIBindings to add the same GVRs moved to Make CRDs and APIBindings conflict on each other for overlapping GVRs #748
    • [ ] add rebinding incl. e2e moved to Add e2e test for rebinding of APIBindings #845
• apis/apiexports: remove status.resourceSchemasInUse #732 ~~ Create APIExportController~~
  • conditions, phases
  • medium-term: prune status.boundResourceSchemas
• APIBinding authorization check that the APIBinding user can access the APIExport objects (for workspace reference) @ncdc apibinding: add admission #653
• Add APIBinding controller, replace inheritFrom with APIBindings #755 Remove inheritFrom from ClusterWorkspaceSpec @ncdc
• Add APIBinding controller, replace inheritFrom with APIBindings #755 Update apiextensions code in kcp & kubernetes to use APIBindings @ncdc
• apibinding demo #763 Contribute to final demo script and recording
• [ ] stretch-goal: admission preventing deletion of APIResourceSchemas that are bound
• prototype4+: Build APIExport virtual apiserver.
• prototype4+ 💣: CR Deletion moved to APIBindings: delete CRs when the APIBinding is deleted #846
• prototype4+: Write APIBinding admission to verify schema compatibility
• prototype4+: Write APIDeployment controller (which rolls out schema updates automatically to APIBindings)
• prototype5+: Write CEL-based migration mechanism as part of APIResourceSchema.

Design Details

API types

Create concrete API types following the sketches in https://docs.google.com/document/d/1k72fgGBNJzbY1Rkxj9_1WksHZR0cNT5OeadCpWuc-zM/edit:

• APIExport
• APIBinding
• APIResourceSchema

APIExportController

• Create/maintain a system:kcp:exported-crds (name TBD) workspace
• Each exported API schema turns into a uniquely named CRD in this workspace
• see https://docs.google.com/document/d/1MhfRk_VV5orvspx1OzU5De7nnE6va9lIpLUKBKnzuWE/edit#heading=h.gcx2dkxb3w1j

APIBindingController

• TODO

apiextensions updates

• Remove inheritFrom logic
• When listing/getting CRDs, combine what's in the current logical cluster with all CRDs associated with all APIBindings for the current workspace
• Deny creation of a CRD in a workspace if the same GVR exists coming from any APIBindings

APIExport virtual apiserver

[pweil-]

Notes from discussion:

evolve APIS

User can A) add a new api, b) deliver a forward compatible change to an existing api export

After this demo, a consumer who leverages these mechanisms will then be a source of input to the "api evolution" epic with additional requirements (i have people using it, i need a non forward change, and i want to do side by side of old api and new api)

[sttts]

Updated the steps:

• no app deployment
• new step with schema update

[fabianvf]

We've been working on making cert-manager multi-cluster aware, if anyone else is working on this issue feel free to reach out (fabianvf on kubernetes slack)

[ncdc]

@sttts I updated the description with more details & design info. PTAL. Can you clarify what "APIImport admission" and "APIDeployment controller" are?

[ncdc]

Status update:

• APIBinding admission is merged apibinding: add admission #653
• WIP: Add APIBinding controller, replace inheritFrom with APIBindings #702 replaces clusterWorkspace.spec.inheritFrom entirely with APIBinding-based extensions
  • The implementation is mostly done. We probably can make changes in follow-up PRs
    • 1 thing that isn't finished / correct is setting the storage versions in the APIBinding's status
  • Not sure on the conditions & reasons I added. Could use a review. Some implementation details probably leak a bit too much.
  • Unit tests are in progress. Working to add more.
  • Replaced the api inheritance e2e with a new apibinding e2e. It tests basic functionality - just that binding works. Things related to testing more involved logic such as rebinding are in unit tests.

[sttts]

Moved everything to issues. Closing the epic for p3.