Travis uses the .travis.yml config from the branch, but tests code from a merge

[Ark-kun]

https://travis-ci.com/kubeflow/pipelines/jobs/182659416
Travis checks out the code correctly:

git clone --depth=50 https://github.com/kubeflow/pipelines.git kubeflow/pipelines
cd kubeflow/pipelines/
git fetch origin +refs/pull/919/merge:1 #Checking out the result of merging PR to master
git checkout -qf FETCH_HEAD

But Travis caches the build configuration (.travis.yml) before the build. If some job fails, and the master is updated, the subsequent runs will use the old build configuration, even though the master have a new one.
So, when someone makes a breaking change that includes a change to the .travis.yml, other PRs may start failing the tests.

[vicaire]

Can this be closed?

[Ark-kun]

This is an external (Travis) issue.