Caching in integration tests caused failures to be ignored

[mugdha-adhav]

What happened?

Integration tests caches output of some steps which resulted in failure to catch errors on PR.

While working on PR, the integration tests were passing.

But when the PR was merged to master branch, the integration tests started failing.

What did you expect to happen?

Integration tests should fail if there are any errors in the test/code files.

[mugdha-adhav]

The integration tests run hack/ci/build.sh script for building the driver image to be used in the integration tests. The script runs docker build command which caches some of the steps.

To disable caching, we can use --no-cache flag in the docker build command.

But instead of directly disabling caching, we should look into how we can enable smart caches where if the changes are detected in the code/tests only then the caching is disabled. Also we should check if there are any other steps/places where we are using cache.

[imuni4fun]

I encountered similar issues when pursuing #137.

In my case, I found that Docker builds would cache bust when i edited *.md files so I created a .dockerignore and added appropriate patterns.

I also found that the initial test that was recommended for checking my dev environment was set up was hiding the fact that pulls were failing. This is because images were cached in my kind cluster's image repo. To address this, I added some make targets and a temporary kind.md file. I am in the process of moving this to a separate PR to simplify the diff for #137.