firejail --join=foobar fails with Error: cannot read /proc file

[simonfxr]

Bug and expected behavior
After updating firejail to 0.9.64.2 trying to join a firejail (either by name or by pid) always fails with:

Switching to pid 3147544, the first child process inside the sandbox
Error: cannot read /proc file

Reproduce
Steps to reproduce the behavior:

1. Run in bash firejail --noprofile --name=foobar /bin/bash
2. Run in bash firejail --join=foobar /bin/bash, it fails with

Switching to pid 3147544, the first child process inside the sandbox
Error: cannot read /proc file

Environment

• Linux distribution: Arch Linux
• Firejail version:

firejail version 0.9.64.2

Compile time support:
	- AppArmor support is enabled
	- AppImage support is enabled
	- chroot support is enabled
	- D-BUS proxy support is enabled
	- file and directory whitelisting support is enabled
	- file transfer support is enabled
	- firetunnel support is enabled
	- networking support is enabled
	- overlayfs support is enabled
	- private-home support is enabled
	- private-cache and tmpfs as user enabled
	- SELinux support is disabled
	- user namespace support is enabled
	- X11 sandboxing support is enabled

Additional context
Other context about the problem like related errors to understand the problem.

Checklist

• The profile (and redirect profile if exists) hasn't already been fixed upstream.
• The program has a profile. (If not, request one in https://github.com/netblue30/firejail/issues/1139)
• I have performed a short search for similar issues (to avoid opening a duplicate).
• If it is a AppImage, --profile=PROFILENAME is used to set the right profile.
• Used LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8 PROGRAM to get english error-messages.
• I'm aware of browser-allow-drm yes/browser-disable-u2f no in firejail.config to allow DRM/U2F in browsers.

[fjthrowaway]

I can confirm the problem, it's still an issue with kernel 5.11
5.10.12 was the last working kernel

[fjthrowaway]

Found a workaround.

Add "lsm=lockdown,yama,apparmor" to /etc/default/grub (GRUB_CMDLINE_LINUX)
then update grub: sudo grub-mkconfig -o /boot/grub/grub.cfg

This will disable bpf (was enabled by default in 5.10.13 on Arch).

[simonfxr]

@fjthrowaway I tried your workaround and is working for me as well, thanks a lot!

[glitsj16]

UPDATE: seems like the workaround is no longer needed on the latest Arch Linux kernel (5.12.15-arch1-1). The cannot read /proc file error is gone. Instead there's a new error thrown - process ID out of range - but joining seems to be working regardless:

$ firejail --noprofile --name=foobar /bin/bash
Parent pid 394123, child pid 394124
Child process initialized in 78.22 ms
[glitsj16@lab16 Downloads]$

$ firejail --join=foobar /bin/bash
Child process initialized in 56.49 ms
error: process ID out of range

Usage:
 ps [options]

 Try 'ps --help <simple|list|output|threads|misc|all>'
  or 'ps --help <s|l|o|t|m|a>'
 for additional help text.

For more details see ps(1).
[glitsj16@lab16 Downloads]$

@simonfxr @fjthrowaway Can you confirm these observations? I realize this isn't a proper fix, but at least disabling bpf is no longer needed (which IMO constitutes a bit of progress).

@smitsohu Anything that can be done to drop that new error message?

[smitsohu]

Anything that can be done to drop that new error message?

Good question. When we print Child process initialized in n.n ms we are mostly done with the sandbox and about to hand over to the sandboxed application, so it looks like bash or a tool invoked by bash prints the error.

Can you reproduce the message with firejail --shell=none --join=foobar /bin/sh or --shell=different_shell?

Maybe the all lower case error message already points in some direction...

[smitsohu]

Maybe the all lower case error message already points in some direction...

Stupid me, that's ps. It is coming from here:
https://gitlab.com/procps-ng/procps/-/blob/v3.3.17/ps/parser.c#L83-L92

Question is why ps is called with an invalid pid.

[glitsj16]

Can you reproduce the message with firejail --shell=none --join=foobar /bin/sh or --shell=different_shell?

Running that doesn't show the error message. So it looks like bash triggers it somehow. Didn't find the time to really debug this further for now. But thanks for providing hints to do that in the very near future.

[ferreum]

I'm still seeing this problem on Arch Linux with kernel 5.16.9-arch1-1 and firejail version 0.9.68, with the same reproduction steps as OP. Same symptoms, only printing this:

Switching to pid 1384, the first child process inside the sandbox
Error: cannot read /proc file

It seems to be related to apparmor. If I remove the security=apparmor kernel parameter I can join other jails just fine.
Interestingly, apparmor profiles don't seem to matter: If I run aa-teardown, or reboot without loading them, joining still results in the same error, as long as security=apparmor is in the kernel command line.

To double-check, aa-status doesn't list anything besides the module being loaded:

# sudo aa-status
apparmor module is loaded.

These are my active LSM when joining doesn't work:

$ cat /sys/kernel/security/lsm
capability,landlock,lockdown,yama,bpf,apparmor

Without security=apparmor, when joining works, only the apparmor module is removed from lsm.

[glitsj16]

These are my active LSM when joining doesn't work:
$ cat /sys/kernel/security/lsm
capability,landlock,lockdown,yama,bpf,apparmor

According to the Arch wiki capability should be omitted. Additionally apparmor needs to come before bpf in the lsm=foo,bar,blah param, at least that's how I understand both the wiki page and kernel documentation. Have you tried adding lsm=landlock,lockdown,yama,apparmor,bpf yet (in that order)?

[ferreum]

Thanks @glitsj16, that helped me fix it! I replaced security=apparmor with lsm=landlock,lockdown,yama,apparmor,bpf and I can now use --join while having apparmor enabled. My /sys/kernel/security/lsm is now capability,landlock,lockdown,yama,apparmor,bpf. Shows it's wise to stick with the Arch wiki instructions.

According to the Arch wiki capability should be omitted.

My mistake was not using lsm= at all, so I didn't state capability. The capability module is always returned by /sys/kernel/security/lsm according to the docs. security=apparmor unfortunately results in the wrong order.

At the risk of going off topic in my defence, the lsm= parameter was where I started out. Eventually I tried security=apparmor instead, because that's what the kernel.org pages described everywhere without mentioning lsm= at all, and things seemed to work fine despite the changed order. Of course I didn't notice the --join issue until much later. I saw an advantage in not restating the whole list; I guess with lsm=, if the default list changes in the future, we would end up excluding modules that would otherwise be loaded? The wiki also only states apparmor has to be the first "major" module, but the kernel.org page linked as reference didn't help me understand which are major modules. It doesn't mention the importance of bpf in particular. Also, I somehow completely overread the Kconfig link…

Anyways, it seems this issue is resolved now, or does anyone else have this issue with --join?

[glitsj16]

I saw an advantage in not restating the whole list; I guess with lsm=, if the default list changes in the future, we would end up excluding modules that would otherwise be loaded?

That's indeed a fair observation. I retested this with a few different kernels on Arch Linux and without explicitly adding the lsm= kernel param AppArmor is not active (even though the module is loaded). As of writing the current default does not include apparmor - regardless of having apparmor=1 security=apparmor:

$ uname -r
5.16.10-arch1-1
$ cat /sys/kernel/security/lsm
capability,landlock,lockdown,yama,bpf

You can always write a simple shell script that checks /sys/kernel/security/lsm on login and notify-send's a warning if apparmor is missing/not in the appropriate order. Very hackish, but sometimes such is life... At least you'll be aware of the situation and can act appropriately.

Anyways, it seems this issue is resolved now, or does anyone else have this issue with --join?

IMO we can close this for now. There's a known workaround and people can always reopen when the problem turns up again.