feat: Setup CI to run tests that require external resources

[reggiemcdonald]

PR to update CI for tests that use external resources

This is tested and working.

I would love to get some feedback on this approach as I'm quite new to setting up circleci.
Are there downsides to not using minikube as in the jaeger-operator repo?

Background

The current CI configuration cannot run tests that rely on any external resources such as databases. In last week's SIG, I was referred to the jaeger-operator for guidance on how to approach the CI changes. The jaeger-operator repo runs the integration tests using minikube and an ubuntu VM under multiple github workflows.

Implementation

This method borrows the idea of using workflows, however, it does not use kubernetes and minikube. Instead, each integration test (one for mongo, one for gocql) is run by a job in a new integration workflow in circleci. The jobs are run using a Makefile target that creates the docker containers needed to run the test (therefore not using any clusters).
.

Changes

• .circleci/config.yml updated to use workflows
• A new workflow was added to .circleci/config.yml for the integration tests that use a matrix of Makefile targets to run the tests
• Makefile targets were created for mongo and gocql
• A helper function was created to move the check for the integration environment variable into a single place
• Since mongo and cassandra take some time to fully start, a shell script was included to wait for the containers. I think this can be made more elegant than it is right now using dockerize

Additional tests can be added by:

1. Modifying Makefile to add a new target
2. Modifying .circleci/config.yml to include the new target in the target matrix

Fixes: #160

\cc @nlehrer @danherr

[Aneurysm9]

This looks good to me. The jaeger-operator tests need to use minikube as they're testing a Kubernetes operator and thus need to be run in the context of a cluster. I think the approach you've taken here is fine for our needs.

[reggiemcdonald]

This looks good to me. The jaeger-operator tests need to use minikube as they're testing a Kubernetes operator and thus need to be run in the context of a cluster. I think the approach you've taken here is fine for our needs.

Thats great! The workflow status isn't getting updated. I think that it might be the result of some circleci settings

Update:

I believe that the pending workflow status is the result of the existing ci workflow being marked as required. This PR renames the old workflow to ci/circleci: build so it will never be reported. I currently do not have the privileges to remove this required check.

[MrAlias]

Overall looks good, I like the approach. Minor cleanup on the bash script is needed.

Do you think it would be possible in the future to only have this run when the gocql or mongo instrumentation contain changes?

[reggiemcdonald]

Overall looks good, I like the approach. Minor cleanup on the bash script is needed.

Do you think it would be possible in the future to only have this run when the gocql or mongo instrumentation contain changes?

Thanks! I was under the impression that the restore_cache step was used to prevent rerunning tests if there weren't any changes. Am I mistaken in that?

EDIT
I see what you mean now. Yes, I think I may be able to write a script to do that

[MrAlias]

Overall looks good, I like the approach. Minor cleanup on the bash script is needed.
Do you think it would be possible in the future to only have this run when the gocql or mongo instrumentation contain changes?

Thanks! I was under the impression that the restore_cache step was used to prevent rerunning tests if there weren't any changes. Am I mistaken in that?

EDIT
I see what you mean now. Yes, I think I may be able to write a script to do that

@reggiemcdonald if you think implementing that kind of selectivity will be a large amount of work I'm in favor of merging this and having that work be a subsequent PR. I'll let you be the one to decide on how you want to split it up.

[reggiemcdonald]

Overall looks good, I like the approach. Minor cleanup on the bash script is needed.
Do you think it would be possible in the future to only have this run when the gocql or mongo instrumentation contain changes?

Thanks! I was under the impression that the restore_cache step was used to prevent rerunning tests if there weren't any changes. Am I mistaken in that?
EDIT
I see what you mean now. Yes, I think I may be able to write a script to do that

@reggiemcdonald if you think implementing that kind of selectivity will be a large amount of work I'm in favor of merging this and having that work be a subsequent PR. I'll let you be the one to decide on how you want to split it up.

I think I came up with a reasonable solution:

Essentially, the makefile targets run a git diff against master and greps for folder names (e.g. gocql). I chose this because it's unknown how many commits a PR may have. So I chose to diff against master and run the integration tests if the module was modified when compared to master. This ensures that integration tests are run for each commit in a given PR that has a modification to the given module (as supposed to just the single commit that modified the module).

Let me know if you like this change. Happy to revert the change or modify the approach 🙂

[MrAlias]

LGTM. Thank you for the contribution!

[reggiemcdonald]

LGTM. Thank you for the contribution!

My pleasure! You'll need to change the required ci check from integration to the other workflows since I dont have access to the repos circleci admin page. Instructions are here

[MrAlias]

LGTM. Thank you for the contribution!

My pleasure! You'll need to change the required ci check from integration to the other workflows since I dont have access to the repos circleci admin page. Instructions are here

@lizthegrey it looks like I don't have access to the needed "branches" section of this repos settings. Is this something you could help with? Giving me access or updating the check that is?

[lizthegrey]

Affirm, I can flip this around.