Development workflow documentation for the current state of the world. #20
Conversation
| order to prepare the environment for running the integration tests, the `pre-integration-test` step must be run in Maven | ||
| on the `resource-managers/kubernetes/integration-tests` module: | ||
|
|
||
| build/mvn pre-integration-test -Pkubernetes -Pkubernetes-integration-tests -pl resource-managers/kubernetes/integration-tests -am |
There was a problem hiding this comment.
once this pre-integration-test profile is run, how do I run the tests themselves? just the k8s ones
There was a problem hiding this comment.
This depends so I wasn't sure how to include it. If just running from the command line, there should be just a maven command for it - this follows the general maven semantics but needing to specify each of the suites means that if we add, rename, or delete suites, we would have to adjust the docs accordingly.
If running in IntelliJ, before running any test in IntelliJ, if any code changes the pre-integration-test has to be run. From there it's just using JUnit to run the test classes.
There was a problem hiding this comment.
Maybe then just a message afterwards saying something like "now run your tests normally, either on the command line with maven or through an IDE like IntelliJ"
| @@ -0,0 +1,45 @@ | |||
| --- | |||
| layout: global | |||
| title: Spark on Kubernetes Development | |||
There was a problem hiding this comment.
at the bottom of the page we should link to the user docs for how to use it as a natural continuation from the dev setup docs
…te-incremental' into k8s-dev-docs
| build/mvn pre-integration-test -Pkubernetes -Pkubernetes-integration-tests -pl resource-managers/kubernetes/integration-tests -am | ||
|
|
||
| Afterwards, the integration tests can be executed with Maven or your IDE. Note that when running tests from an IDE, the | ||
| `pre-integration-test` phase must be run every time the core Kubernetes code changes. When running tests from the |
There was a problem hiding this comment.
by "core Kubernetes code" here you mean the spark-kubernetes integration, not the upstream kubernetes.io code right?
There was a problem hiding this comment.
We should say Spark code in general since people can make changes to core as well.
|
Do we also want to link the proposal, kubernetes/kubernetes#34377 and the JIRA here? |
|
@foxish good idea but I think just the JIRA will suffice since the proposal is attached to it. |
|
Sounds good. Rest LGTM. |
|
@aash addressed clarification, good to merge? |
#20) * Development workflow documentation for the current state of the world. * Address comments. * Clarified code change and added ticket link
#20) * Development workflow documentation for the current state of the world. * Address comments. * Clarified code change and added ticket link
#20) * Development workflow documentation for the current state of the world. * Address comments. * Clarified code change and added ticket link
apache-spark-on-k8s#20) * Development workflow documentation for the current state of the world. * Address comments. * Clarified code change and added ticket link
apache-spark-on-k8s#20) * Development workflow documentation for the current state of the world. * Address comments. * Clarified code change and added ticket link
Note the modules section and the integration tests section isn't too much of a concern since we're changing those pretty soon anyways, at least ideally. The current state of how to run integration tests isn't great and we should be looking into improving that.