feat: create init container for job

[vsukhin]

Pull request description

Create init container for volume initialization

Checklist (choose whats happened)

• breaking change! (describe)
• tested locally
• tested on cluster
• added new dependencies
• updated the docs
• added a test

Breaking changes

Changes

Fixes

[cyrus-mc]

I like this feature but I think it needs to be much more generic. In that the executor POD can be customized in a number of ways.

As an example, for some of my tests I want to ensure that mutating webhook is invoked (ex: Vault secrets). Therefore I would need to add annotations and environment variables to the executor POD.

I am going to take a stab at this but it might be as simple as adding a PodSpec field to the test CRD that allows you to customize the generated Pod.

[exu]

I like this feature but I think it needs to be much more generic. In that the executor POD can be customized in a number of ways.

As an example, for some of my tests, I want to ensure that mutating webhook are invoked (ex: Vault secrets). Therefore I would need to add annotations and environment variables to the executor POD.

I am going to take a stab at this but it might be as simple as adding a PodSpec field to the test CRD that allows you to customize the generated Pod.

Hi, @cyrus-mc thanks for your feedback. I like the idea of adding annotations very much - we could add them to:

• executor - executor would always add those data
• test - tests would add it owns annotations or general pod spec
• tests-executions - test runs could also add it owns

question is if we should pass everything to a single init container or maybe create many init containers for each type (executor, test, execution)? The question also is if doing this in init containers will be enough? (Maybe some users will want to run something after tests?)

@vsukhin ^^^

[exu]

secondly, I would extract the idea of having separate PodSpec as a separate issue as it looks like a quite big topic in terms of possible implementation complexity

[cyrus-mc]

@exu I agree about separating it out. Honestly I wasn't sure what the need for init container is as coded in this PR as the description is quite lacking.

But looking at the changes it is somewhat inline with my request about being able to fully customize the POD the tests run in. And is reason why I commented.

[vsukhin]

Hey, @cyrus-mc. Thank you for your feedback, first of all! Yeah, I agree, that we need to provide more flexibilty in terms of running executor POD in different ways. As it usually happens, we're evolving from simple implementation to more complex and powerful solutions. For sure, you might end up having multiple init containers or none, adding annotations, env vars, etc. The idea behind this pr was to support creation of executor in different programming languages by prefetching test data to pod volume by init container and allow executor container to decide what to do with prefetched data. This is a fisrt step ;)

[exu]

@cyrus-mc we've extracted your suggestions as separate issue here #1008
We'll get into it for 0.11 release (currently we are on the end of 0.10 which should be ok for next week) so we'll get back to it as quick as possible