Make kubeflow/kfctl source of truth for kfctl development; turn down kfctl in kubeflow/kubeflow

[jlewi]

Opening this issue to track migrating development of kfctl from kubeflow/kubeflow to kubeflow/kfctl

@kkasravi What are the next steps here? Are we waiting on E2E tests to be migrated to kubeflow/kfctl?

/assign @kkasravi

[kkasravi]

Yes, I need to add the tests and fix the error you noted where we are actually running e2e on Kubeflow/kubeflow

[jlewi]

@kkasravi Any update on this? When do you think we should migrate development over from kubeflow/kubeflow to kubeflow/kfctl?

It would be great to do this well in advance of trying to cut 0.7.

[jlewi]

@kkasravi any update?

[kkasravi]

@jlewi

There is are several pending PRs that we should merge

• sync with kubeflow v0.6.0-rc.0-82-g489e916f #16 sync with kubeflow/bootstrap
• prow fix to run in the right repo #17 prow fix to build kfctl and run in the right repo
• adds kfctl_k8s_istio config as a pre-submit test #11 add kfctl_k8s_istio config in presubmit tests

#17 I'm working on - should be ready today sometime
#11 is dependent on #17
#16 can be merged

after the above PRs get merged we should switch over.

[jlewi]

Fantastic. Once we start switching over to kubeflow/kfctl; I think we'll need to update the kubeflow/manifests tests?

[kkasravi]

@jlewi yep - we'll need to

• remove kfctl tests in kubeflow/kubeflow that are redundant.
• update manifests/tests/workflows/

[swiftdiaries]

/cc @swiftdiaries

[jlewi]

@kkasravi It looks like #16 and #17 have been merged. Does this mean we are ready to start moving development to kubeflow/kfctl?

[kkasravi]

Yes.

I’ve also moved config over to manifests.
We should probably switch over to that in the same timeframe. I’ll ping you on that PR later today.

[jlewi]

@kkasravi Do you want to send a note to kubeflow-discuss so we can get a heads up to all folks working on kfctl who might have changes in flight? Do we also want to try to identify any PRs in flight that will be impacted?

[jlewi]

/cc @swiftdiaries @gabrielwen

[kkasravi]

done - https://groups.google.com/forum/#!topic/kubeflow-discuss/2T-iiDCb5Q0

[jlewi]

Thanks @kkasravi so can you provide some detail here about the plan

1. When are we going to switch to make kubeflow/kfctl the source of truth?
2. Are we planning on deleting kubeflow/kubeflow/bootstrap to try to make it clear to folks that source of truth is now kubeflow/kfctl
3. Is there going to be some sort of transition period during which PRs could be submitted to both repositories?
   • Could we generate patches from kubeflow/kubeflow and then apply them in kubeflow/kfctl?

A cursory glance shows a number of open PRs still on kubeflow/kubeflow

kubeflow/kubeflow#4110 fix kfctl init
kubeflow/kubeflow#4106 reconcile
kubeflow/kubeflow#4092 fix readme
....

Is there a way to identify all PRs that are modifying files in that directory?

[jlewi]

Here's a recipe for patching commits to kubeflow/kubeflow into kfctl

1. On kubeflow/kubeflow generate a patch for the specified commit e.g

   git format-patch -1 ${COMMIT-SHA}
   

   • This will output a patch file

2. Now apply the patch to kubeflow/kfctl

   git apply --reject -p2 ${PATCH_FILE}
   

   • The reject option means git will skip any patches that can't be applied cleanly

     • This will probably happen for changes to imports
     • Reject files will be spit out

3. Manually fix any remaining merges and then open a PR.

[jlewi]

Here's a list of PRs with kfctl in the name

hub pr list | grep kfctl
   #4115  Support Kubeflow upgrades in kfctl (pt 1)   cla: yes   do-not-merge/hold   size/XL 
   #4092  update kfctl readme   cla: yes   needs-ok-to-test   size/XS 
   #4040  adds apply -f to kfctl   cla: yes   size/L 
   #4017  sync kfctl_gcp_iap.yaml with v0.6.2 config   cla: yes   lgtm   size/M 
   #3949  builds, pushes kfctl to GCS as part of pre and post submits   cla: yes   size/M 

[jlewi]

Here's a proposal (from @abhi-g )

• Until we are ready to cut the v0.7-branch and RC for kfctl, kubeflow/kubeflow remains the source of truth
• When we are ready to cut the v0.7-branch
  • Perform one last sync of kubeflow/kubeflow to kubeflow/kfctl
  • Cut the v0.7-branch and build the RC from kubeflow/kfctl
• At this point kubeflow/kfctl is the source of truth
• Any PRs that end up being committed to kubeflow/kubeflow after that can be ported over as one offs using the git recipe I pasted above

@yanniszark @swiftdiaries @kkasravi @gabrielwen @richardsliu WDYT? Can you thumbs up this comment if this sounds good to you?

[jlewi]

Downgrading to P1 because we will not block 0.7.0 on this.

[jlewi]

Bump to P0. We have moved development over to kubeflow/kfctl so we need to finish getting things moved over as quickly as possible to avoid churn.

The big item is moving over the tests.

• kfctl_basic_auth should probably move from kubeflow/kubeflow to kubeflow/kfctl

• kfctl_go_existing should probably move to kubeflow/kfctl as well

  • We should probably use this as an opportunity to replace ksonnet in the E2E test
  • Related to e2e: refactor existing_arrikto test to use pyfunc kubeflow#4189 - use pyfunc for existing arrikto
  • Related to kfctl E2E test deploying on existing kubernetes cluster; simulated on prem cluster kubeflow#3496 - e2e test for on prem

• I think we can delete kfctl-e2e-minikube

  • We no longer have special logic for minikube; we just handle it as installation on an existing cluster

• The 0.7 kfctl tests should be moved to a v0.7-branch in kubeflow/kfctl

• Python test code related to kfctl should move to kubeflow/kfctl

[jlewi]

@kunmingg Do we need to move over the other click to deploy binaries in
https://github.com/kubeflow/kubeflow/tree/master/bootstrap/cmd ?

[jlewi]

@kunmingg what's your thinking about the source of truth for C2D? Should we move the code in bootstrap into kubeflow/kfctl ?
https://github.com/kubeflow/kubeflow/tree/master/bootstrap

[jlewi]

kubeflow/manifests#613 - is fixed so kubeflow/manifests presubmits are now running E2E tests

I think there are two remaining issues to close this out.

1. Move kfctl py code into kubeflow/kfctl #77 - Finish migration of the python code
2. Migrate C2D code from kubeflow/kubeflow/bootstrap into kubeflow/kfctl #141 - Migrate C2D code from kubeflow/kubeflow to kubeflow/kfctl

[jlewi]

The only remaining work is moving for the C2D code. That has its own P2 bug (#141) so I'm going to go ahead and close this issue.