Fix condition for k8 container completion check #27502
Conversation
|
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
|
| while not self.container_is_running(pod=pod, container_name=container_name): | ||
| while self.container_is_running(pod=pod, container_name=container_name): |
There was a problem hiding this comment.
same fix was suggest and merged in #23883 but It was reverted in #24479
so just trying the same thing again is probably not going to work?
cc @akakakakakaa as the original author
There was a problem hiding this comment.
Hi, thanks, yeah I'm aware. I opened this PR as a Draft (but wasn't aware that still pinged anyone) to see if I could run the CI tests.
The original PR was reverted due to failing tests, but unfortunately the CI logs and the slack conversations have both fallen victim to the retention. I'm still figuring out the testing flow locally but haven't found any failing tests yet, so must be missing something (as a new contributor).
|
The test failure in CI, https://github.com/apache/airflow/actions/runs/3393106966/jobs/5643930560, does not seem to be caused by this change. It's failing on |
|
restarted it. Likely flake |
|
Yep. Passed. Is it ready for review? Can you mark it as such if it is please ? |
Since the original PR failed due to missing tests, I have been trying to identify which those failing tests could be. However, I haven't been able to fail any tests yet. All tests succeed. I noticed that one code path wasn't properly tested, so I'm refactoring the tests to cover that. That doesn't answer why the tests failed in main for the previous PR though.. @potiuk Do you know if there is a way to trigger more tests in CI than what was run for this PR? I successfully ran |
| @@ -810,7 +816,11 @@ def test_pod_template_file_with_full_pod_spec(self): | |||
| assert k.pod.spec.containers[0].env == [k8s.V1EnvVar(name="env_name", value="value")] | |||
| assert result == {"hello": "world"} | |||
|
|
|||
| def test_full_pod_spec(self): | |||
| @pytest.mark.parametrize( | |||
There was a problem hiding this comment.
Adding support for this required me to move from a unittest.TestCase to pytest. This required a couple of other fixes, but is worth it in my mind, since other test classes are using pytest.
| """ | ||
| ).strip() | ||
| assert any(line.startswith(expected_line) for line in cm.output) | ||
| expected_line = ( |
There was a problem hiding this comment.
Due to the different log format of caplog, I decided to reformat this to a regex search instead of the any(... logic. This was easier to do with this string format instead of the multiline.
As described in #26796, the condition for checkin whether the k8 container is completed is incorrectly reversed. This was initially discovered and fixed in #23883, but ultimately reverted due to failing tests.
This PR attempts to address the failing tests by extending the tests to cover this path.
closes: #26796
related: #23883, #24479
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.