Gradle clean failing after a failed gradle check, folders created by Docker under 'root' user

[reta]

Signed-off-by: Andriy Redko andriy.redko@aiven.io

Description

the composeUp needs preProcessFixture to be executed before, otherwise the folders:

distribution/docker/build/logs/1
distribution/docker/build/logs/2

won't exists and will be created by Docker under root user:

drwxrwxr-x 3 user.user user.user 4.0K Nov 29 15:40 classes
drwxrwxr-x 4 user.user user.user 4.0K Nov 30 10:03 docker
drwxrwxr-x 3 user.user user.user 4.0K Nov 29 15:38 forbidden-apis-config
drwxrwxr-x 3 user.user user.user 4.0K Nov 29 15:40 generated
drwxrwxr-x 2 user.user user.user 4.0K Nov 29 15:45 heapdump
drwxr-xr-x 4 root         root         4.0K Dec 14 11:24 logs
drwxrwxr-x 2 user.user user.user 4.0K Nov 29 15:44 markers
drwxrwxrwx 2 user.user user.user 4.0K Dec 13 09:20 repo
drwxrwxr-x 4 user.user user.user 4.0K Nov 29 15:45 reports

Issues Resolved

Closes #1675

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[opensearch-ci-bot]

Can one of the admins verify this patch?

[reta]

@dblock @VachaShah this one needs backport to 1.x, just checked it and it has the same issue

[saratvemulapalli]

@reta this seems like ./gradlew clean should always fail.
But folks see this only at random times, I dont quite understand why.

[reta]

thanks for looking @saratvemulapalli

@reta this seems like ./gradlew clean should always fail. But folks see this only at random times, I dont quite understand why.

Not necessary, it essentially depends if Gradle needs to execute task preProcessFixture or it is not ("up to date"). So you could easily get into this situation by running a build, then cleaning some projects (gradle :distribution:docker:clean) and than running some tasks right after (gradle :distribution:docker:composeUp) - reproducible for me 100%. I don't know exactly how @VachaShah run into that but it is not difficult.

PS: Once the folders are created under root, the gradle clean will always fail, but usually folders are pre-created before by preProcessFixture task.

But this is only one particular issue for distribution/docker module, there are others (but may be the cause is the same).

[opensearch-ci-bot]

✅   Gradle Check success c8144b6229479edb3f55418e64653577a291f70b
Log 1487

Reports 1487

[reta]

Same issue for remote-clusters module

[saratvemulapalli]

thanks for looking @saratvemulapalli

@reta this seems like ./gradlew clean should always fail. But folks see this only at random times, I dont quite understand why.

Not necessary, it essentially depends if Gradle needs to execute task preProcessFixture or it is not ("up to date"). So you could easily get into this situation by running a build, then cleaning some projects (gradle :distribution:docker:clean) and than running some tasks right after (gradle :distribution:docker:composeUp) - reproducible for me 100%. I don't know exactly how @VachaShah run into that but it is not difficult.

PS: Once the folders are created under root, the gradle clean will always fail, but usually folders are pre-created before by preProcessFixture task.

But this is only one particular issue for distribution/docker module, there are others (but may be the cause is the same).

Makes sense, thanks for the explanation.

[reta]

Thanks @saratvemulapalli , did minor refactor to use configure properly, had weird Gradle behaviour for remote-clusters module, should be all good now

[opensearch-ci-bot]

✅   Gradle Check success 605cb6e8a7154668190c4ed8c39a52fe68cfd06b
Log 1489

Reports 1489

[VachaShah]

Thank you @reta for taking this up!

[opensearch-ci-bot]

❌   Gradle Check failure b662d8131b836b7d1999feaab92a337462d01e40
Log 1491

Reports 1491

[opensearch-ci-bot]

❌   Gradle Check failure 7aeb0630522fccee5110cb35e4c463ffc0c14cdc
Log 1506

Reports 1506

[opensearch-ci-bot]

✅   Gradle Check success 5bea661
Log 1543

Reports 1543

[reta]

Thanks a lot @dblock , I will create backport to 1.x on Monday