Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

flaky chrony + resolved SELinux AVC denial #751

Closed
cgwalters opened this issue Feb 19, 2021 · 4 comments
Closed

flaky chrony + resolved SELinux AVC denial #751

cgwalters opened this issue Feb 19, 2021 · 4 comments
Labels
kind/bug

Comments

@cgwalters
Copy link
Member

See coreos/rpm-ostree#2598 (comment)

This is some sort of flake though because I'm at just a 1/8 failure rate:

walters@toolbox /v/s/w/b/fcos> cosa kola run --multiply 8 --parallel 3 ext.config.podman.rootless-systemd
⚠️  Skipping kola test pattern "fcos.internet":
⚠️  https://github.com/coreos/coreos-assembler/pull/1478
⚠️  Skipping kola test pattern "podman.workflow":
⚠️  https://github.com/coreos/coreos-assembler/pull/1478
kola --denylist-test fcos.internet --denylist-test podman.workflow -p qemu-unpriv --output-dir tmp/kola run --multiply 8 --parallel 3 ext.config.podman.rootless-systemd
2021-02-19T19:18:41Z kola: Found non-executable file with shebang: install-wrappers.sh
2021-02-19T19:18:41Z kola: Found non-executable file with shebang: itest-bare-unit.sh
=== RUN   ext.config.podman.rootless-systemd3
=== RUN   ext.config.podman.rootless-systemd4
=== RUN   ext.config.podman.rootless-systemd5
=== RUN   ext.config.podman.rootless-systemd6
=== RUN   ext.config.podman.rootless-systemd7
=== RUN   ext.config.podman.rootless-systemd0
=== RUN   ext.config.podman.rootless-systemd1
=== RUN   ext.config.podman.rootless-systemd2
--- PASS: ext.config.podman.rootless-systemd1 (80.07s)
--- PASS: ext.config.podman.rootless-systemd3 (84.20s)
--- PASS: ext.config.podman.rootless-systemd7 (91.51s)
--- FAIL: ext.config.podman.rootless-systemd2 (18.15s)
        harness.go:963: Cluster failed starting machines: machine "a78ec6c4-5732-4d33-b7f4-83e1a1313800" failed basic checks: Found SELinux AVC denials:
Feb 19 19:20:11 qemu0 audit[843]: AVC avc:  denied  { read } for  pid=843 comm="chronyd" name="stub-resolv.conf" dev="tmpfs" ino=1402 scontext=system_u:system_r:chronyd_t:s0 tcontext=system_u:object_r:systemd_resolved_var_run_t:s0 tclass=lnk_file permissive=0
Feb 19 19:20:11 qemu0 audit[843]: AVC avc:  denied  { read } for  pid=843 comm="chronyd" name="stub-resolv.conf" dev="tmpfs" ino=1402 scontext=system_u:system_r:chronyd_t:s0 tcontext=system_u:object_r:systemd_resolved_var_run_t:s0 tclass=lnk_file permissive=0
--- PASS: ext.config.podman.rootless-systemd0 (83.73s)
--- PASS: ext.config.podman.rootless-systemd5 (77.11s)
--- PASS: ext.config.podman.rootless-systemd6 (80.80s)
--- PASS: ext.config.podman.rootless-systemd4 (64.34s)
@cgwalters cgwalters changed the title failed basic checks: Found SELinux AVC denials: Feb 19 19:20:11 qemu0 audit[843]: AVC avc: denied { read } for pid=843 comm="chronyd" name="stub-resolv.conf" dev="tmpfs" ino=1402 scontext=system_u:system_r:chronyd_t:s0 tcontext=system_u:object_r:systemd_resolved_var_run_t:s0 tclass=lnk_file permissive=0 flaky chrony + resolved SELinux failure Feb 19, 2021
@cgwalters cgwalters changed the title flaky chrony + resolved SELinux failure flaky chrony + resolved SELinux AVC denial Feb 19, 2021
@cgwalters
Copy link
Member Author

Theory: this happens when chronyd.service starts before NM gives resolved the DNS servers - i.e. we don't have the right label during early bootup.

@cgwalters
Copy link
Member Author

This is likely fixed by systemd/systemd@7b87bec

@jlebon
Copy link
Member

jlebon commented Feb 19, 2021

We carry a temporary hack in FCOS until we get that systemd patch: coreos/fedora-coreos-config@b9a2c39. Likely just need to order it before chronyd? From coreos/fedora-coreos-config#780 (comment), we should be able to drop that hack entirely once we move to f34. (Which... I'll add a note to that effect in #704 so we don't forget.)

@jlebon jlebon mentioned this issue Feb 19, 2021
Closed
cgwalters added a commit to cgwalters/coreos-assembler that referenced this issue Feb 22, 2021
@cgwalters
kola: Add --no-default-checks
a70546a 
This will allow us to e.g. bypass SELinux denials when we need
to, e.g.  coreos/fedora-coreos-tracker#751

The intention is that for FCOS releases we would drive this declaratively
in the pipeline, something like a:

```
kola:
  no-default-checks: 33.20210201.3.0
```

to automatically suppress checks for builds with that version number.
@cgwalters cgwalters mentioned this issue Feb 22, 2021
Merged
cgwalters added a commit to cgwalters/coreos-assembler that referenced this issue Feb 23, 2021
@cgwalters
kola: Add --no-default-checks
2bf1b35 
This will allow us to e.g. bypass SELinux denials when we need
to, e.g.  coreos/fedora-coreos-tracker#751

The intention is that for FCOS releases we would drive this declaratively
in the pipeline, something like a:

```
kola:
  no-default-checks: 33.20210201.3.0
```

to automatically suppress checks for builds with that version number.
openshift-merge-robot pushed a commit to coreos/coreos-assembler that referenced this issue Feb 23, 2021
@cgwalters @openshift-merge-robot
kola: Add --no-default-checks
0d07f4e 
This will allow us to e.g. bypass SELinux denials when we need
to, e.g.  coreos/fedora-coreos-tracker#751

The intention is that for FCOS releases we would drive this declaratively
in the pipeline, something like a:

```
kola:
  no-default-checks: 33.20210201.3.0
```

to automatically suppress checks for builds with that version number.
jlebon added a commit to jlebon/fedora-coreos-pipeline that referenced this issue Mar 2, 2021
@jlebon
Temporarily use --no-default-checks
7cec02b 
We're hitting SELinux flakes in the releases:
coreos/fedora-coreos-tracker#751

Working on a better way to allow known SELinux failures, but for now
let's just use the new `--no-default-checks` to unblock releases.
@jlebon jlebon mentioned this issue Mar 2, 2021
Merged
jlebon added a commit to coreos/fedora-coreos-pipeline that referenced this issue Mar 2, 2021
@jlebon
Temporarily use --no-default-checks
74a78a3 
We're hitting SELinux flakes in the releases:
coreos/fedora-coreos-tracker#751

Working on a better way to allow known SELinux failures, but for now
let's just use the new `--no-default-checks` to unblock releases.
@dustymabe
Copy link
Member

We carry a temporary hack in FCOS until we get that systemd patch: coreos/fedora-coreos-config@b9a2c39. Likely just need to order it before chronyd? From coreos/fedora-coreos-config#780 (comment), we should be able to drop that hack entirely once we move to f34. (Which... I'll add a note to that effect in #704 so we don't forget.)

The workaround was reverted in coreos/fedora-coreos-config@474c87b so we're no longer carrying the workaround and have the new systemd with the fix.

Closing this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@cgwalters @dustymabe @jlebon