Support make check
#9614
Closed
Support make check
#9614
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
On modern Linux systems there is no need to set a specific `/proc/sys/kernel/core_pattern`, the `coredumpctl dump` command can be used to extract the core using the specific name. By removing the need to set the core_pattern zloop.sh can now be run with normal user priviledges. Previous work has already been done to allow ztest to run without any of the kmods being loaded. Additionally, cleanup a few remaining shellcheck warnings in the zloop.sh script. Allow options to be passed as environment variables. Print a message when running for a fixed duration. Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Add support for the traditional `make check` target. When invoked,
the following test wrappers will be run:
* scripts/zloop.sh - Performs a 10 minute run using the default
options. This test does not require escaleted priviledges.
* scripts/zfs-test.sh - Perform a full run of the ZFS Test Suite.
This normally will take several hours to complete and requires:
1) The user sudo configured to not require a password.
2) The ZFS kernel modules must already be loaded.
3) All required system utilities are installed.
4) /var/tmp/ has at least 12G of available free space.
5) When running in-tree the zfs-helper.sh symlinks exist.
If these requirements are not met `make check` may skip the
ZTS run entirely, or individual test cases may fail.
Functionally, this commit updates the top-level Makefile.am
to set TESTS and TESTS_ENVIRONMENT for `make check` to run.
Improves the environment sanity checks performed by zfs-tests.sh
before starting testing. This includes verifing the kmods are
loaded (it is no longer done automatically), the required utilities
are available, and passwordless sudo is configured.
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Codecov Report
@@ Coverage Diff @@
## master #9614 +/- ##
==========================================
+ Coverage 79.19% 79.21% +0.01%
==========================================
Files 418 418
Lines 123531 123531
==========================================
+ Hits 97828 97849 +21
+ Misses 25703 25682 -21
Continue to review full report at Codecov.
|
|
IIRC some users reported loss of data running the ZTS with their personal/production pools attached/imported (which in itself is a bad idea), we should probably document this feature as "possibly destructive", maybe print a warning or ask for confirmation before execution. |
|
On November 23, 2019 12:28 am, Brian Behlendorf wrote:
### Motivation and Context
Traditionally `make check` is the mechanism used to invoke the
test suite for a C program built with autoconf/automake. Supporting
this functionality is desirable since it makes it easier for users and
new ZFS developers to get started working with the code.
Depends on #9613. Posted for review and discussion, but blocked
until #9613 is approved and merged.
### Description
Add support for the traditional `make check` target. When invoked,
the following test wrappers will be run:
* scripts/zloop.sh - Performs a 10 minute run using the default
options. This test does not require escaleted priviledges.
that seems fine for 'make check'
* scripts/zfs-test.sh - Perform a full run of the ZFS Test Suite.
This normally will take several hours to complete and requires:
1) The user sudo configured to not require a password.
2) The ZFS kernel modules must already be loaded.
3) All required system utilities are installed.
4) /var/tmp/ has at least 12G of available free space.
5) When running in-tree the zfs-helper.sh symlinks exist.
this seems a bit too much for my taste. 'make check' is run by default
in Debian packaging, for example. it is more akin to what a CI would do
(in Debian/Ubuntu terms, an autopkgtest that tests the built
package+environment, not just the build output). while it would be
trivial to override in Debian+derivative packaging, my guess is that
other distros use 'make check' similarly?
…
If these requirements are not met `make check` may skip the
ZTS run entirely, or individual test cases may fail.
Functionally, this commit updates the top-level Makefile.am
to set TESTS and TESTS_ENVIRONMENT for `make check` to run.
Improves the environment sanity checks performed by zfs-tests.sh
before starting testing. This includes verifing the kmods are
loaded (it is no longer done automatically), the requi
### How Has This Been Tested?
In a properly configured VM will all of the required dependencies
installed:
```
sh autogen.sh
./configure
make -s -j$(nproc)
sudo ./scripts/zfs.sh`
sudo ./scripts/zfs-helpers.sh -iv`
make check
...
Executing zloop.sh for 600 seconds.
11/22 14:33:24 /home/behlendo/src/git/zfs/bin/ztest -G -VVVVV -m 2 -r 0 -R 2 -v 4 -a 9 -T 86 -P 20 -s 512m -f /var/tmp/zloop-run
11/22 14:35:15 /home/behlendo/src/git/zfs/bin/ztest -G -VVVVV -m 2 -r 10 -R 2 -v 4 -a 12 -T 12 -P 11 -s 512m -f /var/tmp/zloop-run
11/22 14:37:25 /home/behlendo/src/git/zfs/bin/ztest -G -VVVVV -m 2 -r 0 -R 2 -v 5 -a 9 -T 34 -P 17 -s 512m -f /var/tmp/zloop-run
11/22 14:38:38 /home/behlendo/src/git/zfs/bin/ztest -G -VVVVV -m 1 -r 8 -R 2 -v 4 -a 9 -T 54 -P 13 -s 512m -f /var/tmp/zloop-run
11/22 14:40:20 /home/behlendo/src/git/zfs/bin/ztest -G -VVVVV -m 2 -r 0 -R 1 -v 2 -a 9 -T 43 -P 19 -s 512m -f /var/tmp/zloop-run
11/22 14:41:59 /home/behlendo/src/git/zfs/bin/ztest -G -VVVVV -m 2 -r 0 -R 2 -v 4 -a 12 -T 83 -P 36 -s 512m -f /var/tmp/zloop-run
zloop finished, 0 crashes found
PASS: scripts/zloop.sh
Test: /home/behlendo/src/git/zfs/tests/zfs-tests/tests/functional/libzfs/setup (run as root) [00:01] [PASS]
Test: /home/behlendo/src/git/zfs/tests/zfs-tests/tests/functional/libzfs/many_fds (run as root) [00:00] [PASS]
...
PASS: scripts/zfs-tests.sh
==================
All 2 tests passed
==================
```
Submitted to the CI to verify `zfs-tests.sh` on other distributions.
### Types of changes
<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Performance enhancement (non-breaking change which improves efficiency)
- [x] Code cleanup (non-breaking change which makes code smaller or more readable)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] Documentation (a change to man pages or other documentation)
### Checklist:
<!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [x] My code follows the ZFS on Linux [code style requirements](https://github.com/zfsonlinux/zfs/blob/master/.github/CONTRIBUTING.md#coding-conventions).
- [x] I have updated the documentation accordingly.
- [x] I have read the [**contributing** document](https://github.com/zfsonlinux/zfs/blob/master/.github/CONTRIBUTING.md).
- [x] I have added [tests](https://github.com/zfsonlinux/zfs/tree/master/tests) to cover my changes.
- [x] All new and existing tests passed.
- [x] All commit messages are properly formatted and contain [`Signed-off-by`](https://github.com/zfsonlinux/zfs/blob/master/.github/CONTRIBUTING.md#signed-off-by).
You can view, comment on, or merge this pull request online at:
#9614
-- Commit Summary --
* Update zloop.sh to use coredumpctl
* Enable `make check` target
-- File Changes --
M Makefile.am (6)
M scripts/common.sh.in (1)
M scripts/zfs-helpers.sh (47)
M scripts/zfs-tests.sh (69)
M scripts/zloop.sh (81)
-- Patch Links --
https://github.com/zfsonlinux/zfs/pull/9614.patch
https://github.com/zfsonlinux/zfs/pull/9614.diff
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
#9614
Motivation and Context
Traditionally make check is the mechanism used to invoke the
test suite for a C program built with autoconf/automake. Supporting
this functionality is desirable since it makes it easier for users and
new ZFS developers to get started working with the code.
Depends on #9613. Posted for review and discussion, but blocked
until #9613 is approved and merged.
Description
Add support for the traditional make check target. When invoked,
the following test wrappers will be run:
• scripts/zloop.sh - Performs a 10 minute run using the default
options. This test does not require escaleted priviledges.
• scripts/zfs-test.sh - Perform a full run of the ZFS Test Suite.
This normally will take several hours to complete and requires:
1. The user sudo configured to not require a password.
2. The ZFS kernel modules must already be loaded.
3. All required system utilities are installed.
4. /var/tmp/ has at least 12G of available free space.
5. When running in-tree the zfs-helper.sh symlinks exist.
If these requirements are not met make check may skip the
ZTS run entirely, or individual test cases may fail.
Functionally, this commit updates the top-level Makefile.am
to set TESTS and TESTS_ENVIRONMENT for make check to run.
Improves the environment sanity checks performed by zfs-tests.sh
before starting testing. This includes verifing the kmods are
loaded (it is no longer done automatically), the requi
How Has This Been Tested?
In a properly configured VM will all of the required dependencies
installed:
sh autogen.sh
./configure
make -s -j$(nproc)
sudo ./scripts/zfs.sh`
sudo ./scripts/zfs-helpers.sh -iv`
make check
...
Executing zloop.sh for 600 seconds.
11/22 14:33:24 /home/behlendo/src/git/zfs/bin/ztest -G -VVVVV -m 2 -r 0 -R 2 -v 4 -a 9 -T 86 -P 20 -s 512m -f /var/tmp/zloop-run
11/22 14:35:15 /home/behlendo/src/git/zfs/bin/ztest -G -VVVVV -m 2 -r 10 -R 2 -v 4 -a 12 -T 12 -P 11 -s 512m -f /var/tmp/zloop-run
11/22 14:37:25 /home/behlendo/src/git/zfs/bin/ztest -G -VVVVV -m 2 -r 0 -R 2 -v 5 -a 9 -T 34 -P 17 -s 512m -f /var/tmp/zloop-run
11/22 14:38:38 /home/behlendo/src/git/zfs/bin/ztest -G -VVVVV -m 1 -r 8 -R 2 -v 4 -a 9 -T 54 -P 13 -s 512m -f /var/tmp/zloop-run
11/22 14:40:20 /home/behlendo/src/git/zfs/bin/ztest -G -VVVVV -m 2 -r 0 -R 1 -v 2 -a 9 -T 43 -P 19 -s 512m -f /var/tmp/zloop-run
11/22 14:41:59 /home/behlendo/src/git/zfs/bin/ztest -G -VVVVV -m 2 -r 0 -R 2 -v 4 -a 12 -T 83 -P 36 -s 512m -f /var/tmp/zloop-run
zloop finished, 0 crashes found
PASS: scripts/zloop.sh
Test: /home/behlendo/src/git/zfs/tests/zfs-tests/tests/functional/libzfs/setup (run as root) [00:01] [PASS]
Test: /home/behlendo/src/git/zfs/tests/zfs-tests/tests/functional/libzfs/many_fds (run as root) [00:00] [PASS]
...
PASS: scripts/zfs-tests.sh
==================
All 2 tests passed
==================
Submitted to the CI to verify zfs-tests.sh on other distributions.
Types of changes
• [ ] Bug fix (non-breaking change which fixes an issue)
• [*] New feature (non-breaking change which adds functionality)
• [ ] Performance enhancement (non-breaking change which improves efficiency)
• [*] Code cleanup (non-breaking change which makes code smaller or more
readable)
• [ ] Breaking change (fix or feature that would cause existing functionality
to change)
• [ ] Documentation (a change to man pages or other documentation)
Checklist:
• [*] My code follows the ZFS on Linux code style requirements.
• [*] I have updated the documentation accordingly.
• [*] I have read the contributing document.
• [*] I have added tests to cover my changes.
• [*] All new and existing tests passed.
• [*] All commit messages are properly formatted and contain Signed-off-by.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
You can view, comment on, or merge this pull request online at:
#9614
Commit Summary
• Update zloop.sh to use coredumpctl
• Enable `make check` target
File Changes
• M Makefile.am (6)
• M scripts/common.sh.in (1)
• M scripts/zfs-helpers.sh (47)
• M scripts/zfs-tests.sh (69)
• M scripts/zloop.sh (81)
Patch Links:
• https://github.com/zfsonlinux/zfs/pull/9614.patch
• https://github.com/zfsonlinux/zfs/pull/9614.diff
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.*
|
Good idea, I'll see about adding some additional safe guards.
Yes, I wondered about that. My thought at the time was this wouldn't effect most CI setups since the ZTS would likely be skipped entirely since it does require some customization of the environment (passwordless sudo, kmods loaded, plus some additional mandatory packages). Perhaps we should simply disable the ZTS entirely from |
lundman
reviewed
Dec 12, 2019
| @@ -455,7 +464,7 @@ if [ "$(id -u)" = "0" ]; then | |||
| fi | |||
|
|
|||
| if [ "$(sudo whoami)" != "root" ]; then | |||
| fail "Passwordless sudo access required." | |||
| skip "Passwordless sudo access required." | |||
There was a problem hiding this comment.
Doesn't this ask for a password, if sudo is not configured? I think you can use sudo -n true and return 0 is sudo-with-password is OK.
|
On December 12, 2019 6:37 am, Jorgen Lundman wrote:
lundman commented on this pull request.
> @@ -455,7 +464,7 @@ if [ "$(id -u)" = "0" ]; then
fi
if [ "$(sudo whoami)" != "root" ]; then
- fail "Passwordless sudo access required."
+ skip "Passwordless sudo access required."
Doesn't this ask for a password, if sudo is not configured? I think you can use `sudo -n true` and return 0 is sudo-with-password is OK.
without prior authentication, password required:
$ sudo -nl
sudo: a password is required
$ echo $?
1
now after authentication (with cached credentials, passwordles would
work the same all the time):
$ sudo -nl
Matching Defaults entries for USER on HOST:
env_reset, mail_badpass, secure_path=/usr/local/sbin\:/usr/local/bin\:/usr/sbin\:/usr/bin\:/sbin\:/bin
User USER may run the following commands on HOST:
(ALL : ALL) ALL
$ echo $?
0
man sudo:
If the -l option was specified without a command, sudo will exit with a value of 0 if the user is allowed to run sudo and they authenticated successfully (as required by the security policy).
the output is of course uninteresting, and can be thrown away. but, this
variant also supports testing individual commands:
man sudo:
If a command is specified with the -l option, the exit value will only be 0 if the command is permitted by the security policy, otherwise it will be 1.
which would allow easy integration (e.g., test case specifies which
commands it needs to run as root, test helper checks all those are
available to user via passwordless sudo before starting the test) with
fine-grained permissions. probably overkill, but maybe worth mentioning
anyway ;)
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Traditionally
make checkis the mechanism used to invoke thetest suite for a C program built with autoconf/automake. Supporting
this functionality is desirable since it makes it easier for users and
new ZFS developers to get started working with the code.
Depends on #9613. Posted for review and discussion, but blocked
until #9613 is approved and merged.
Description
Add support for the traditional
make checktarget. When invoked,the following test wrappers will be run:
scripts/zloop.sh - Performs a 10 minute run using the default
options. This test does not require escaleted priviledges.
scripts/zfs-test.sh - Perform a full run of the ZFS Test Suite.
This normally will take several hours to complete and requires:
If these requirements are not met
make checkmay skip theZTS run entirely, or individual test cases may fail.
Functionally, this commit updates the top-level Makefile.am
to set TESTS and TESTS_ENVIRONMENT for
make checkto run.Improves the environment sanity checks performed by zfs-tests.sh
before starting testing. This includes verifing the kmods are
loaded (it is no longer done automatically), the requi
How Has This Been Tested?
In a properly configured VM will all of the required dependencies
installed:
Submitted to the CI to verify
zfs-tests.shon other distributions.Types of changes
Checklist:
Signed-off-by.