Issue with KFP 0.1.20 and TFX in GCP

[ianhellstrom]

I'm running a TFX pipeline in Kubeflow (0.4.1) in GCP. Prior to KFP 0.1.20 it worked fine, but now the initial step where it reads data from BigQuery fails with:

Could not get main container logs: Error: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods \"my-model-h2nfc-1334252523\" not found","reason":"NotFound","details":{"name":"my-model-h2nfc-1334252523","kind":"pods"},"code":404}

Except ship the pipeline file with 0.1.20 I did not change anything in the setup.

[jessiezcc]

does it repro with kubeflow 0.5?

[ianhellstrom]

Since our Kubeflow 0.5 cluster does not work properly due to #1355, I cannot say. We're stuck on 0.4.1 for the foreseeable future.

[jessiezcc]

@ianhellstrom, #1355 is closed, can you pls try the new CLI?

[jessiezcc]

close since no update.